Document content analysis and identification method and device, electronic equipment and storage medium

Abstract

The invention provides a document content analysis and identification method and device, electronic equipment and a storage medium, and relates to the technical field of document processing. According to the method, a to-be-analyzed document is input into a preset detection model comprising a controllable receiving module, a preset integration algorithm and a global-to-local design structure, and content features corresponding to at least one page in the to-be-analyzed document are obtained according to the controllable receiving module; according to the content features and a preset integration algorithm, obtaining corresponding integration features; according to the method and the device, the to-be-analyzed document is obtained, the integrated features are input into a global-to-local design structure, a detection positioning mode for obtaining the document content in the to-be-analyzed document is obtained, and finally the document content in the to-be-analyzed document is identified and obtained based on the detection positioning mode, so that the structural integrity of PDF document analysis and identification is optimized, and the PDF document analysis and identification accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of document processing technology, and in particular to a document content parsing and recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] PDF (Portable Document Format) documents are a file format that maintains consistent formatting across different devices and operating systems. With the advancement of information technology, PDF documents are widely used in government, finance, healthcare, and legal sectors to store important documents such as contracts, reports, accounts, and technical files. However, because some PDF documents contain scanned copies or compressed images, they lack a directly recognizable text layer and require parsing and recognition before further editing or use.

[0003] Current PDF document parsing and recognition solutions typically employ recognition software based on OCR (Optical Character Recognition) technology, such as Tesseract, ABBYY, and PaddleOCR, to recognize characters in the content of PDF documents, and then construct editable structured documents, including text layers, based on the recognized characters.

[0004] However, existing OCR-based recognition software typically only recognizes text and struggles to accurately segment document elements such as titles, body text, tables, headers, and footers. Especially for complex layouts with multi-column layouts, nested tables, or image annotations, it often fails to reconstruct the semantic logic of the original document. Therefore, current PDF document parsing and recognition solutions suffer from poor structural integrity and low accuracy. Summary of the Invention

[0005] The main objective of this application is to propose a document content parsing and recognition method, apparatus, electronic device, and storage medium, aiming to optimize the structural integrity of PDF document parsing and recognition and improve the parsing and recognition accuracy.

[0006] In a first aspect, the present invention provides a document content parsing and recognition method, comprising: The document to be parsed is input into a preset detection model, wherein the preset detection model includes: a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; The controllable receiving module obtains the content features corresponding to at least one page of the document to be parsed. Based on the content features and the preset integration algorithm, the corresponding integration features are obtained; Based on the integrated features and the global-to-local design structure, the document content in the document to be parsed is identified and obtained.

[0007] In an optional implementation, inputting the document to be parsed into a preset detection model includes: The document to be parsed is split into multiple pages to be parsed; The multiple pages to be parsed are input into the preset detection model in the form of an image matrix.

[0008] In an optional implementation, obtaining the content features corresponding to at least one page of the document to be parsed based on the controllable receiving module includes: Based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, obtain the content features corresponding to each preset dilation rate in one or more pages to be parsed.

[0009] In an optional implementation, the step of obtaining content features corresponding to each preset dilation rate in one or more pages to be parsed, based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, includes: Based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, preliminary features corresponding to each preset dilation rate in one or more of the pages to be parsed are obtained. Based on the preset normalization layer and the preliminary features, the normalized features of each preliminary feature are obtained respectively; Based on the preset activation function and the normalized features, the activation features corresponding to each normalized feature are obtained respectively, and the activation features are the content features.

[0010] In an optional implementation, identifying and obtaining the document content in the document to be parsed based on the integrated features and the global-to-local design structure includes: The integrated features are input into the global-to-local design structure to obtain detection results corresponding to one or more pages to be parsed. The detection results include: formula detection results, title detection results, text detection results, and table detection results. Based on the formula detection results, the title detection results, the text detection results, and the table detection results, determine the comprehensive detection result corresponding to the page to be parsed; Based on the comprehensive detection results, multiple positioning boxes in the page to be parsed are determined; Based on the positioning boxes and the global-to-local design structure, the local content corresponding to each positioning box in each page to be parsed is identified and obtained; Based on the local content and the preset layout rules, determine the document content after layout.

[0011] In an optional implementation, before inputting the document to be parsed into the preset detection model, the method further includes: Receive multiple document parsing requests submitted by multiple users, each document parsing request including: parsing instructions and one or more document objects; According to a preset queuing rule, multiple document objects are queued to determine the order in which they are input into the preset detection model. The preset queuing rule includes: priority rule, reception time rule, and document type rule.

[0012] In an optional implementation, the parsing instruction further includes: specifying the identifier of the page; The step of obtaining the content features corresponding to at least one page in the document to be parsed based on the controllable receiving module includes: According to the parsing instructions and the current parsing mode, the controllable receiving module is used to parse the pages to be parsed corresponding to the identifiers of one or more specified pages to obtain the corresponding content features. The current parsing mode includes: parallel parsing mode or independent parsing mode.

[0013] In a second aspect, the present invention provides a document content parsing and recognition device, comprising: The input module is used to input the document to be parsed into a preset detection model, wherein the preset detection model includes: a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; The acquisition module is used to acquire the content features corresponding to at least one page in the document to be parsed based on the controllable receiving module; An integration module is used to obtain corresponding integration features based on the content features and a preset integration algorithm; The identification module is used to identify and obtain the document content in the document to be parsed based on the integrated features and the global-to-local design structure.

[0014] Thirdly, the present invention provides an electronic device comprising: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method described in the first aspect above.

[0015] Fourthly, the present invention provides a computer-readable storage medium on which a computer program is stored, the computer program being executed by a processor to perform the steps of the method described in the first aspect above.

[0016] The beneficial effects of this application are: The document content parsing and recognition method provided in this application includes: inputting the document to be parsed into a preset detection model including a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; the controllable receiving module acquires content features for at least one page of the document to be parsed, and combines the acquired content features with the preset integration algorithm to obtain corresponding integrated features; then inputting the integrated features into the global-to-local design structure; and finally, recognizing and acquiring the document content in the document to be parsed based on the output result of the global-to-local design structure. In this embodiment, a preset detection model, including a controllable receiving module, a preset integration algorithm, and a global-to-local design structure, processes the document to be parsed. This involves first obtaining the content features of at least one page of the document from the controllable receiving module, then combining these content features with the preset integration algorithm to obtain corresponding integrated features, and finally inputting these integrated features into the global-to-local design structure to obtain a detection and localization method for the document content. Finally, the document content is identified and obtained based on this detection and localization method. The controllable receiving module flexibly extracts and integrates features at multiple scales and granularities during the document content parsing and recognition process. The global-to-local design structure performs hierarchical perception from global context (full-page scale) to sub-block regions (medium scale) and then to local semantic information during the document content parsing and recognition process, thereby optimizing the structural integrity of PDF document parsing and recognition and improving the accuracy of PDF document parsing and recognition. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This application provides a schematic flowchart of a document content parsing and recognition method according to an embodiment of the present application. Figure 2 A flowchart illustrating a document content parsing and recognition method according to another embodiment of this application; Figure 3 A flowchart illustrating a document content parsing and recognition method provided in another embodiment of this application; Figure 4 A flowchart illustrating a document content parsing and recognition method provided in another embodiment of this application; Figure 5 A schematic diagram of the structure of a document content parsing and recognition device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0023] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0024] Current PDF document parsing and recognition solutions typically employ recognition software based on OCR (Optical Character Recognition) technology, such as Tesseract, ABBYY, and PaddleOCR, to recognize characters in the content of PDF documents, and then construct editable structured documents, including text layers, based on the recognized characters.

[0025] However, existing OCR-based recognition software typically only recognizes text and struggles to accurately segment document elements such as titles, body text, tables, headers, and footers. Especially for complex layouts with multi-column layouts, nested tables, or image annotations, it often fails to reconstruct the semantic logic of the original document. The extracted content after parsing and recognition may suffer from layout errors, incorrect order, formatting changes, and garbled characters. Therefore, current PDF document parsing and recognition solutions suffer from poor structural integrity and low accuracy.

[0026] To address the aforementioned issues, the main objective of this application is to propose a document content parsing and recognition method that aims to optimize the structural integrity of PDF document parsing and recognition and improve the accuracy of parsing and recognition.

[0027] Figure 1 This is a flowchart illustrating a document content parsing and recognition method according to an embodiment of this application. The executing entity of this method can be, for example, a computer, a server, or other device with computing power, but is not limited thereto. Please refer to... Figure 1 The method includes: S101. Input the document to be parsed into the preset detection model, wherein the preset detection model includes: a controllable receiving module, a preset integration algorithm, and a global-to-local design structure.

[0028] For example, the document to be parsed mentioned above can refer to a PDF document whose content is a scanned copy or an image-compressed PDF document. Such PDF documents usually lack a directly identifiable text layer and need to be parsed and recognized before further editing or use. However, the specific type and content format of the document to be parsed are not limited to this. It is understood that when the document to be parsed is a scanned copy or an image-compressed PDF document, the above-mentioned input of the document to be parsed into the preset detection model can be equivalent to inputting the document into the preset detection model in image form, but is not limited to this.

[0029] The aforementioned preset detection model can be, for example, a target detection model based on deep learning technology, whose core architecture can be adapted to the characteristic of significant differences in element scale in document images.

[0030] The aforementioned controllable receiving module can be used, for example, to implement the feature extraction function of a preset detection model. This controllable receiving module can flexibly capture visual features of different granularities and scales in the input image.

[0031] The aforementioned preset integration algorithm can, for example, be used to fuse and recombine multiple sets of features extracted by the controllable receiving module. However, the specific fusion and recombination process and method are not specifically limited here, but can be adjusted and determined according to the actual situation.

[0032] The aforementioned global-to-local design structure can be used, for example, to define a progressive analysis and recognition process for a pre-defined detection model, from understanding the overall global layout context of the page to focusing on local semantic details.

[0033] The aforementioned controllable receiving module, preset integration algorithm, and global-to-local design structure work together to enable the preset detection model to perform end-to-end positioning and preliminary classification of various elements (such as text, titles, tables, etc.) within a document page.

[0034] S102. Obtain the content features corresponding to at least one page of the document to be parsed based on the controllable receiving module.

[0035] For example, the content features corresponding to at least one page in the document to be parsed can refer to the feature acquisition of the content in the document to be parsed page by page according to the default pagination method of the document to be parsed. Specifically, the controllable receiving module can acquire the content features corresponding to at least one page in the document to be parsed by extracting features from the content features corresponding to at least one page in the document to be parsed through a pre-set weight-sharing convolutional layer w in the controllable receiving module. The kernel size of the convolutional layer can be represented as k, and the specific value of k can be adjusted and determined according to the actual situation, without limitation here.

[0036] It is understandable that the aforementioned controllable receiving module acquiring the content features corresponding to at least one page of the document to be parsed can mean that the controllable receiving module acquires the content features corresponding to each page of the document to be parsed sequentially according to page number order, or it can mean that the controllable receiving module determines one or more pages of the document to be parsed based on user instructions or preset rules (such as rules for determining one or more pages corresponding to the most core and important content in the document, but not limited to this), and acquires the content features of that page or pages. Of course, the specific method used to determine the page of the document to be parsed from which the controllable receiving module acquires content features is not limited to the examples above, but can be adjusted and determined according to actual needs.

[0037] S103. Based on the above content features and the preset integration algorithm, obtain the corresponding integration features.

[0038] For example, the aforementioned content features can include features at multiple scales and granularities. The aforementioned preset integration algorithm can refer to, for example, a feature concatenation algorithm, but is not limited to this. Taking the preset integration algorithm as an example of a feature concatenation algorithm, the corresponding integrated features are obtained based on the aforementioned content features and the preset integration algorithm. For example, it can refer to concatenating multiple feature vectors output by the aforementioned controllable receiving module, each corresponding to a different preset inflation rate, along the channel dimension. This concatenation process aggregates discrete multi-scale feature information into a unified, richer feature tensor, i.e., the aforementioned integrated features. This operation enables subsequent networks to simultaneously access and utilize information from different receptive fields, providing a crucial data foundation for accurately identifying document elements with large size spans (such as tables spanning the entire page and single-line headings).

[0039] Of course, when the above-mentioned preset integration algorithm is another type of algorithm, the above-mentioned content features and preset integration algorithm may also obtain the corresponding integration features through methods different from the examples above, and are not limited to the examples above.

[0040] S104. Based on the above integration features and the above global-to-local design structure, identify and obtain the document content in the above document to be parsed.

[0041] For example, the above-mentioned integrated features and the above-mentioned global-to-local design structure identify and obtain the document content in the document to be parsed. For example, the above-mentioned integrated features can be used as input values ​​to the above-mentioned global-to-local design structure. The above-mentioned global-to-local design structure performs upsampling, downsampling, fusion and other operations on the input integrated features, thereby constructing a multi-level structure containing high-resolution details (e.g., corresponding to local) and rich semantics (e.g., corresponding to global).

[0042] Specifically, after receiving integrated feature input, the aforementioned global-to-local design structure can, for example, at the "global" level, preliminarily predict the bounding box positions and categories of various document elements (such as titles, paragraphs, tables, images, etc.) based on the feature context of the entire page, forming preliminary detection results. Subsequently, at the "local" level, the structure can perform more refined feature analysis or substructure recognition on candidate regions. For example, for regions identified as tables, their internal cell structure can be further analyzed. For text regions, their image slices can be fed into models such as OCR (Optical Character Recognition) for character recognition. Finally, the system integrates the recognition results from all levels, organizes and sorts them according to preset layout rules, and outputs the structured document content. This content typically includes the type, position, text information, and logical order of elements, and can be organized into markup languages ​​(such as Markdown) or other structured data formats for further processing or use.

[0043] Of course, the above-mentioned "global" and "local" levels are a possible assumption, used to indicate that the above global to local design structure can make predictions at different levels for features of different scales and granularities, and then obtain recognition results at different levels based on the predictions at different levels. The recognition results at all levels are combined, and the structured document content is output according to the preset layout rules. The specific number of levels is not limited to the two levels of global and local.

[0044] The document content parsing and recognition method provided in this application includes: inputting the document to be parsed into a preset detection model including a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; the controllable receiving module acquires content features for at least one page of the document to be parsed, and combines the acquired content features with the preset integration algorithm to obtain corresponding integrated features; then inputting the integrated features into the global-to-local design structure; and finally, recognizing and acquiring the document content in the document to be parsed based on the output result of the global-to-local design structure. In this embodiment, a preset detection model, including a controllable receiving module, a preset integration algorithm, and a global-to-local design structure, processes the document to be parsed. This involves first obtaining the content features of at least one page of the document from the controllable receiving module, then combining these content features with the preset integration algorithm to obtain corresponding integrated features, and finally inputting these integrated features into the global-to-local design structure to obtain a detection and localization method for the document content. Finally, the document content is identified and obtained based on this detection and localization method. The controllable receiving module flexibly extracts and integrates features at multiple scales and granularities during the document content parsing and recognition process. The global-to-local design structure performs hierarchical perception from global context (full-page scale) to sub-block regions (medium scale) and then to local semantic information during the document content parsing and recognition process, thereby optimizing the structural integrity of PDF document parsing and recognition and improving the accuracy of PDF document parsing and recognition.

[0045] Optionally, in the above Figure 1 Based on the embodiments, the above-mentioned document to be parsed is input into a preset detection model, including: The document to be parsed is split into multiple pages to be parsed.

[0046] The multiple pages to be parsed are input into the preset detection model in the form of an image matrix.

[0047] Furthermore, based on the above embodiments, the controllable receiving module acquires content features corresponding to at least one page of the document to be parsed, including: Based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, obtain the content features corresponding to each preset dilation rate in one or more of the pages to be parsed.

[0048] For example, the aforementioned preset expansion rates can be expressed as follows: , where d is a set of multiple preset expansion rates, These are the aforementioned preset dilation rates. The specific value of each preset dilation rate can be adjusted and determined according to actual needs, and is not limited here. The aforementioned convolutional layer can refer to, for example, the preset weight-sharing convolutional layer w in the above embodiment. The kernel size of the aforementioned convolutional layer can refer to, for example, the kernel size k of the convolutional layer in the above embodiment. The aforementioned image matrix corresponding to the page to be parsed is the input value of inputting the multiple pages to be parsed into the aforementioned preset detection model in the form of an image matrix. The content features corresponding to each of the aforementioned preset dilation rates in one or more of the aforementioned pages to be parsed can be represented, for example, as follows: Among them, the above This refers to the set of content features corresponding to each of the aforementioned preset inflation rates in one or more of the aforementioned pages to be parsed. That is, the content feature corresponding to each of the above preset expansion rates, where the preset expansion rate... The corresponding content features are Preset expansion rate The corresponding content features are ...preset expansion rate The corresponding content features are .

[0049] Figure 2 This is a schematic flowchart of a document content parsing and recognition method provided in another embodiment of this application. Further, as... Figure 2 As shown, based on the above embodiments, the method of obtaining content features corresponding to each preset dilation rate in one or more pages to be parsed, according to multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, includes: S201. Based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, obtain preliminary features corresponding to each preset dilation rate in one or more of the pages to be parsed.

[0050] For example, the preliminary feature corresponding to each of the preset inflation rates in one or more of the aforementioned pages to be parsed can be represented as follows: ,in The above image matrix corresponds to the page to be parsed. This refers to the kernel size value of the above convolutional layer. For multiple preset expansion rates (i=1, 2...n). For dilated convolution operations, for a preset dilation rate Its corresponding preliminary characteristics are For the preset expansion rate Its corresponding preliminary characteristics are ...for the preset expansion rate Its corresponding preliminary characteristics are .

[0051] S202. Based on the preset normalization layer and the above preliminary features, obtain the normalized features of each of the above preliminary features respectively.

[0052] For example, the aforementioned preset normalization layer can be represented as a layer normalization function. Of course, the above This is just an illustration; the actual layer normalization function structure can be adjusted and determined according to specific circumstances. For example, this layer normalization function can be used to avoid gradient explosion during document content parsing and recognition by a pre-defined detection model.

[0053] Based on this, the normalized feature of each of the aforementioned preliminary features can be expressed as follows:

[0054] S203. Based on the preset activation function and the above-mentioned normalized features, obtain the activation features corresponding to each normalized feature. The above-mentioned activation features are the above-mentioned content features.

[0055] For example, the aforementioned preset activation function can be expressed as follows: Of course, the above This is just an illustration; the actual preset activation function structure can be adjusted and determined according to specific circumstances. For example, this preset activation function can be used to add nonlinear relationships within a preset detection model.

[0056] Based on this, the activation feature corresponding to each of the above normalized features can be represented as follows:

[0057] The above content features are represented as follows: Then we have:

[0058] Furthermore, based on the examples above, the corresponding integration features obtained according to the aforementioned content features and the preset integration algorithm in the foregoing embodiments can be represented as follows:

[0059] Among them, the above This refers to the aforementioned integration features, the aforementioned For the feature concatenation function corresponding to the above feature concatenation algorithm, when the above preset integration algorithm is another type of algorithm, this... The function can also be changed to other functions; no specific restrictions are imposed here.

[0060] Figure 3This application provides a flowchart illustrating a document content parsing and recognition method according to another embodiment. Optionally, please refer to... Figure 3 Based on the foregoing embodiments, the above-mentioned integrated features and global-to-local design structure, in order to identify and obtain the document content in the document to be parsed, include: S301. Input the above-mentioned integrated features into the above-mentioned global-to-local design structure to obtain one or more detection results corresponding to the above-mentioned pages to be parsed. The above-mentioned detection results include: formula detection results, title detection results, text detection results, and table detection results.

[0061] For example, the integrated features mentioned above can carry multi-scale information of the page. After being input into the global-to-local design structure, the detection head (or classification network) inside the global-to-local design structure will perform inference based on these features.

[0062] For example, a global-to-local design structure might begin by analyzing each potential region on the integrated features, predicting its probability of belonging to a preset category (such as formulas, titles, plain text, tables, etc.), and simultaneously regressing the boundary coordinates of that region, thus outputting preliminary detection results. For instance, the formula detection result indicates the location and confidence level of the mathematical formula region, the title detection result corresponds to different levels of titles, the text detection result covers body paragraphs, and the table detection result locates the table region. These results collectively constitute a preliminary structured understanding of the page content, but the specific element categories included in the detection results are not limited to the examples above and can be adjusted according to the training objectives of the preset detection model.

[0063] S302. Based on the above formula detection results, the above title detection results, the above text detection results, and the above table detection results, determine the comprehensive detection result corresponding to the above page to be parsed.

[0064] For example, the above formula detection results, above title detection results, above text detection results, and above table detection results determine the comprehensive detection results corresponding to the above page to be parsed. For example, it can refer to the fusion and post-processing of the above formula detection results, above title detection results, above text detection results, and above table detection results to form a unified and conflict-free description of page elements.

[0065] Specifically, the comprehensive detection results for the page to be parsed can be determined based on a preset comprehensive algorithm (such as non-maximum suppression algorithm, but not limited to it) and preset intersection-union ratio and confidence scores of the predicted boxes to eliminate duplicate or overlapping detections of the same physical region, retaining the optimal predicted box for each element. Simultaneously, logical verification can be performed based on the spatial relationships between different categories of detection results; for example, ensuring that a region identified as a table does not contain independent text detection boxes. After the above filtering and optimization, all detection boxes and their determined category labels are summarized to generate a structured list, i.e., the comprehensive detection results for the page to be parsed. This result clearly defines the category and precise location of all identified elements on the page.

[0066] S303. Based on the above comprehensive test results, identify multiple positioning boxes in the page to be parsed.

[0067] For example, the comprehensive detection results described above identify multiple location boxes in the page to be parsed. This could refer to a set of geometric information extracted from the structured information contained in the comprehensive detection results to guide the accurate extraction of subsequent content. The location boxes refer to the geometric parameters of the bounding rectangle corresponding to each confirmed document element (such as a title, a table, or a piece of text) in the comprehensive detection results.

[0068] Specifically, for example, after parsing the comprehensive detection results, the coordinate description of each valid element (such as its top-left vertex coordinates, width value, and height value, but not limited to these) can be obtained, and a corresponding bounding box can be determined accordingly. Finally, a set of multiple bounding boxes is output, where each bounding box precisely defines the region of an independent semantic module in the page image, providing input for subsequently obtaining the specific content of each region.

[0069] S304. Based on the above positioning boxes and the above global-to-local design structure, identify and obtain the local content corresponding to each of the above positioning boxes in each page to be parsed.

[0070] For example, the aforementioned positioning boxes and the aforementioned global-to-local design structure respectively identify and obtain the local content corresponding to each of the aforementioned positioning boxes in each page to be parsed. For example, it can refer to the process of performing refined content recognition on the image region defined by each of the aforementioned positioning boxes, dominated by the aforementioned global-to-local design structure.

[0071] In one possible implementation, the aforementioned positioning boxes and the aforementioned global-to-local design structure respectively identify and obtain the local content corresponding to each of the aforementioned positioning boxes in each page to be parsed. For example, based on the geometric parameters of the aforementioned positioning boxes, the corresponding region sub-images can be extracted one by one from the image of the page to be parsed. After that, the aforementioned global-to-local design structure can schedule or integrate the corresponding recognition function units for processing according to the category information (such as text, tables, etc.) associated with the aforementioned positioning boxes.

[0072] For example, for text-based regions, optical character recognition (OCR) can be used to extract the text information. For table-based regions, table line detection and cell content extraction can be performed. Based on this, a corresponding, deep-analyzed content result is generated for each of the above-mentioned bounding boxes, i.e., the aforementioned local content.

[0073] S305. Based on the above partial content and preset layout rules, determine the document content after layout.

[0074] For example, the aforementioned partial content and preset layout rules determine the document content after layout. This could refer to the process of reorganizing and encoding all the aforementioned partial content based on the original logical and visual structure of the document.

[0075] In practical applications, the process of reorganizing and encoding all the aforementioned partial content based on the original logic and visual structure of the document can, for example, begin by sorting and combining all the aforementioned partial content and their corresponding position and category information according to the logic defined in the aforementioned preset layout rules (e.g., mainly determining the content order based on the vertical coordinates of the aforementioned positioning boxes and establishing hierarchical relationships based on element categories, but not limited to this). Ultimately, a data entity carrying the complete document structure and semantic content can be obtained, such as a markup language document or a structured data object. This data entity is the document content after the aforementioned layout, thus completing the transformation from page image information to ordered, reusable document data.

[0076] Of course, the above are just possible examples, and the actual implementation methods for each step are not limited to the examples above.

[0077] Figure 4 This is a flowchart illustrating a document content parsing and recognition method according to another embodiment of the present application. Optionally, as shown... Figure 4 As shown above, in the aforementioned Figure 1 Based on the embodiments, before inputting the document to be parsed into the preset detection model, the following steps are also included: S401. Receive multiple document parsing requests submitted by multiple users, each of the above document parsing requests including: parsing instructions and one or more document objects.

[0078] For example, receiving multiple document parsing requests submitted by multiple users can, for instance, refer to supporting concurrent access by multiple users through a request access and management module or a request access and management interface, and receiving multiple document parsing requests submitted by multiple users based on the support for concurrent access by multiple users.

[0079] For example, the above parsing instructions may refer to the above... Figure 1 The user instructions in the embodiment, such as the parsing instructions, can be used to indicate the parsing urgency and parsing page range of the corresponding document object, but are not limited thereto.

[0080] The document object mentioned above is used to indicate the corresponding document to be parsed.

[0081] S402. According to the preset queuing rules, the multiple document objects are queued to determine the order in which they are input into the preset detection model. The preset queuing rules include: priority rules, receiving time rules, and document type rules.

[0082] For example, the priority rule mentioned above could be a rule that determines the order in which document objects are input into the preset detection model based on the parsing urgency of the corresponding document objects indicated by the parsing instruction. For example, document objects with higher urgency are queued first and input into the preset detection model first.

[0083] The aforementioned receiving time rule could be, for example, a rule that determines the order in which document objects are input into the aforementioned preset detection model based on the receiving time of each document object. For instance, document objects with earlier receiving times are queued up first and input into the aforementioned preset detection model first.

[0084] The document type rules mentioned above could be rules that determine the order in which document objects are input into the preset detection model based on the type of each document object. For example, document objects with important or confidential document types are queued up first and input into the preset detection model first.

[0085] It is understandable that the aforementioned priority rules, receiving time rules, and document type rules can be used to determine the order in which multiple document objects are input into the preset detection model. If multiple document objects have the same priority, the order in which they are input into the preset detection model can be determined according to the document type rules. If multiple document objects also have the same type, the order in which they are input into the preset detection model can be determined according to the receiving time rules. Of course, the above is only one possible way of queuing document objects and is not limited to this in practice. Furthermore, the specific rules of the aforementioned priority rules, receiving time rules, and document type rules can be adjusted and determined according to the actual situation, and are not restricted here.

[0086] Furthermore, in the above Figure 4 Based on the embodiments, the above parsing instructions also include: specifying the identifier of the page.

[0087] The aforementioned controllable receiving module acquires content features corresponding to at least one page of the document to be parsed, including: Based on the above parsing instructions and the current parsing mode, the above controllable receiving module is used to parse the pages to be parsed corresponding to the identifiers of one or more specified pages to obtain the corresponding content features. The above current parsing mode includes: parallel parsing mode or independent parsing mode.

[0088] For example, the parallel parsing mode mentioned above can refer to a parsing mode in which multiple pages to be parsed are parsed simultaneously. This mode is applicable to scenarios with sufficient hardware resources, simple page structures, and few element contents. The independent parsing mode mentioned above can refer to a parsing mode in which a single page to be parsed is parsed independently. This mode is applicable to scenarios with limited hardware resources, complex page structures, and many element contents. However, it is not limited to these specific scenarios.

[0089] Figure 5 This is a schematic diagram of a document content parsing and recognition device according to an embodiment of this application. This device can execute the aforementioned document content parsing and recognition method. The device can be integrated into devices with computing capabilities, such as computers and servers. Figure 5 As shown, the device may include: The input module 510 is used to input the document to be parsed into a preset detection model, wherein the preset detection model includes: a controllable receiving module, a preset integration algorithm, and a global-to-local design structure.

[0090] The acquisition module 520 is used to acquire the content features corresponding to at least one page of the document to be parsed based on the controllable receiving module.

[0091] The integration module 530 is used to obtain the corresponding integration features based on the above content features and the preset integration algorithm.

[0092] The identification module 540 is used to identify and obtain the document content in the document to be parsed based on the above-mentioned integrated features and the above-mentioned global-to-local design structure.

[0093] The document content parsing and recognition method provided in this application includes: inputting the document to be parsed into a preset detection model including a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; the controllable receiving module acquires content features for at least one page of the document to be parsed, and combines the acquired content features with the preset integration algorithm to obtain corresponding integrated features; then inputting the integrated features into the global-to-local design structure; and finally, recognizing and acquiring the document content in the document to be parsed based on the output result of the global-to-local design structure. In this embodiment, a preset detection model, including a controllable receiving module, a preset integration algorithm, and a global-to-local design structure, processes the document to be parsed. This involves first obtaining the content features of at least one page of the document from the controllable receiving module, then combining these content features with the preset integration algorithm to obtain corresponding integrated features, and finally inputting these integrated features into the global-to-local design structure to obtain a detection and localization method for the document content. Finally, the document content is identified and obtained based on this detection and localization method. The controllable receiving module flexibly extracts and integrates features at multiple scales and granularities during the document content parsing and recognition process. The global-to-local design structure performs hierarchical perception from global context (full-page scale) to sub-block regions (medium scale) and then to local semantic information during the document content parsing and recognition process, thereby optimizing the structural integrity of PDF document parsing and recognition and improving the accuracy of PDF document parsing and recognition.

[0094] Optionally, the input module 510 is specifically used to split the document to be parsed into multiple pages to be parsed. These multiple pages are then input into the preset detection model in the form of an image matrix.

[0095] Optionally, the acquisition module 520 is specifically used to acquire content features corresponding to each of the preset dilation rates in one or more of the pages to be parsed, based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed.

[0096] Optionally, the acquisition module 520 is specifically used to acquire preliminary features corresponding to each of the preset dilation rates in one or more pages to be parsed, based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed. Based on a preset normalization layer and the preliminary features, normalized features are acquired for each of the preliminary features. Based on a preset activation function and the normalized features, activation features corresponding to each normalized feature are acquired; these activation features are the content features.

[0097] Optionally, the aforementioned recognition module 540 is specifically used to input the integrated features into the global-to-local design structure to obtain detection results corresponding to one or more of the pages to be parsed. These detection results include: formula detection results, title detection results, text detection results, and table detection results. Based on the formula detection results, title detection results, text detection results, and table detection results, a comprehensive detection result corresponding to the page to be parsed is determined. Based on the comprehensive detection result, multiple positioning boxes in the page to be parsed are determined. Based on the positioning boxes and the global-to-local design structure, the local content corresponding to each positioning box in each page of the page to be parsed is identified and obtained. Based on the local content and preset layout rules, the document content after layout is determined.

[0098] Optionally, the document content parsing and recognition device further includes a receiving module, configured to receive multiple document parsing requests submitted by multiple users, each document parsing request including a parsing instruction and one or more document objects. The multiple document objects are queued according to a preset queuing rule to determine the order in which they are input into the preset detection model, wherein the preset queuing rule includes a priority rule, a receiving time rule, and a document type rule.

[0099] Optionally, the above parsing instructions may also include: specifying the identifier of the page.

[0100] The aforementioned acquisition module 520 is specifically used to parse the pages to be parsed corresponding to the identifiers of one or more specified pages according to the aforementioned parsing instructions and the current parsing mode, and to obtain the corresponding content features. The aforementioned current parsing mode includes: parallel parsing mode or independent parsing mode.

[0101] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0102] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be a computer, server, or other device with computing power, as described above. Figure 6 As shown, the device 600 includes: The processor 610, storage medium 620, and bus 630 are connected in communication via bus 630.

[0103] The storage medium 620 stores machine-readable instructions that can be executed by the processor 610. When the electronic device is running, the processor 610 executes the aforementioned machine-readable instructions to perform the aforementioned document content parsing and recognition method.

[0104] It should be understood that, Figure 6 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown. Figure 6 The components shown can be implemented using hardware, software, or a combination thereof.

[0105] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the document content parsing and recognition method described in the above method embodiments.

[0106] Computer-readable storage media can be electronic storage devices such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, computer-readable storage media includes non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code that performs any of the method steps described above. This program code can be read from or written to one or more computer program exhibits. The program code can be compressed, for example, in a suitable form.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program exhibits according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0108] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0109] If the functionality is implemented as a software module and sold or used as an independent exhibit, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software exhibit. This computer software exhibit is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The above description is merely a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural transformations made based on the inventive concept of this application and the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A document content parsing and recognition method, characterized in that, include: The document to be parsed is input into a preset detection model, wherein the preset detection model includes: a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; The controllable receiving module obtains the content features corresponding to at least one page of the document to be parsed. Based on the content features and the preset integration algorithm, the corresponding integration features are obtained; Based on the integrated features and the global-to-local design structure, the document content in the document to be parsed is identified and obtained.

2. The document content parsing and recognition method according to claim 1, characterized in that, The step of inputting the document to be parsed into the preset detection model includes: The document to be parsed is split into multiple pages to be parsed; The multiple pages to be parsed are input into the preset detection model in the form of an image matrix.

3. The document content parsing and recognition method according to claim 2, characterized in that, The step of obtaining the content features corresponding to at least one page in the document to be parsed based on the controllable receiving module includes: Based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, obtain the content features corresponding to each preset dilation rate in one or more pages to be parsed.

4. The document content parsing and recognition method according to claim 3, characterized in that, The step of obtaining content features corresponding to each preset dilation rate in one or more pages to be parsed, based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, includes: Based on multiple preset dilation rates, the kernel size of the convolutional layer, and the image matrix corresponding to the page to be parsed, preliminary features corresponding to each preset dilation rate in one or more of the pages to be parsed are obtained; Based on the preset normalization layer and the preliminary features, the normalized features of each preliminary feature are obtained respectively; Based on the preset activation function and the normalized features, the activation features corresponding to each normalized feature are obtained respectively, and the activation features are the content features.

5. The document content parsing and recognition method according to claim 2, characterized in that, The step of identifying and obtaining document content in the document to be parsed based on the integrated features and the global-to-local design structure includes: The integrated features are input into the global-to-local design structure to obtain detection results corresponding to one or more pages to be parsed. The detection results include: formula detection results, title detection results, text detection results, and table detection results. Based on the formula detection results, the title detection results, the text detection results, and the table detection results, determine the comprehensive detection result corresponding to the page to be parsed; Based on the comprehensive detection results, multiple positioning boxes in the page to be parsed are determined; Based on the positioning boxes and the global-to-local design structure, the local content corresponding to each positioning box in each page to be parsed is identified and obtained; Based on the local content and the preset layout rules, determine the document content after layout.

6. The document content parsing and recognition method according to claim 1, characterized in that, Before inputting the document to be parsed into the preset detection model, the process also includes: Receive multiple document parsing requests submitted by multiple users, each document parsing request including: parsing instructions and one or more document objects; According to a preset queuing rule, multiple document objects are queued to determine the order in which they are input into the preset detection model. The preset queuing rule includes: priority rule, reception time rule, and document type rule.

7. The document content parsing and recognition method according to claim 6, characterized in that, The parsing instructions also include: specifying the identifier of the page; The step of obtaining the content features corresponding to at least one page in the document to be parsed based on the controllable receiving module includes: According to the parsing instructions and the current parsing mode, the controllable receiving module is used to parse the pages to be parsed corresponding to the identifiers of one or more specified pages to obtain the corresponding content features. The current parsing mode includes: parallel parsing mode or independent parsing mode.

8. A document content parsing and recognition device, characterized in that, include: The input module is used to input the document to be parsed into a preset detection model, wherein the preset detection model includes: a controllable receiving module, a preset integration algorithm, and a global-to-local design structure; The acquisition module is used to acquire the content features corresponding to at least one page in the document to be parsed based on the controllable receiving module; An integration module is used to obtain corresponding integration features based on the content features and a preset integration algorithm; The identification module is used to identify and obtain the document content in the document to be parsed based on the integrated features and the global-to-local design structure.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1-7.

Images