Document processing method and device, computer equipment, storage medium and program product

Abstract

The invention discloses a document processing method and device, computer equipment, a storage medium and a program product, and belongs to the technical field of computers. The method comprises the following steps: acquiring a to-be-processed document; the to-be-processed document is in a portable document format; analyzing the to-be-processed document page by page to obtain page content of each page, and performing semantic block recognition on each page content to obtain a plurality of semantic blocks; according to a semantic extraction strategy of modal perception, respectively extracting semantic elements from each semantic block; and according to a predefined semantic block structural body, performing standardized packaging on the semantic elements to generate a plurality of structural body instances for associating original contents in the to-be-processed document in the semantic task. The flexibility and accuracy of dynamic partitioning can be improved, and then the accuracy of subsequent semantic tasks is improved.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a document processing method, apparatus, computer equipment, storage medium and program product. Background Technology

[0002] With the widespread use of PDF (Portable Document Format) documents in various fields, more and more semantic processing tasks, such as intelligent retrieval, knowledge extraction, and question answering generation, are beginning to rely on a deep, structured understanding of the content of PDF documents.

[0003] Traditional PDF content processing methods generally employ fixed-chunk segmentation to understand documents. This method divides documents according to a fixed number of words or lines, ignoring the structural and semantic differences between different content types within the document. It also fails to capture the document's inherent semantic structure, easily leading to semantic fragmentation and structural disorder, thus affecting the effectiveness of subsequent tasks such as information retrieval, knowledge extraction, and natural language generation. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the related art. To this end, this application proposes a document processing method, apparatus, computer device, storage medium, and program product that can improve the flexibility and accuracy of dynamic segmentation, thereby enhancing the accuracy of subsequent semantic tasks.

[0005] Firstly, this application provides a document processing method, the method comprising:

[0006] Obtain the document to be processed; the document to be processed is in a portable document format.

[0007] The document to be processed is parsed page by page to obtain the page content of each page, and semantic block recognition is performed on the content of each page to obtain multiple semantic blocks;

[0008] Based on the modality-aware semantic extraction strategy, semantic elements are extracted from each semantic block.

[0009] According to the predefined semantic chunk structure, the semantic elements are standardized and encapsulated to generate multiple structure instances, which are used to associate the original content in the document to be processed in the semantic task.

[0010] According to the document processing method provided in this application, page content is obtained through page-by-page parsing, and semantic block recognition is performed on each page content to obtain multiple semantic blocks. This achieves efficient identification and block segmentation of page content based on the inherent characteristics of different content types without introducing complex learning models. It also avoids the problem of traditional rule templates being unable to adapt to complex page layouts, thus improving the parsing capability for multimodal mixed-format documents. Furthermore, based on a modality-aware semantic extraction strategy, semantic elements are extracted from each semantic block, improving the accuracy of semantic element extraction and avoiding the problems of semantic missingness or chaotic extraction in unstructured documents. The semantic elements are standardized and encapsulated according to a predefined semantic block structure, establishing a unified, indexable, and callable semantic data expression format. This not only enhances the structural alignment capability between different semantic types but also preserves the semantic information and layout context of the original content in the generated structure instances, possessing high searchability and vectorization capabilities. This better supports downstream tasks such as semantic retrieval, question answering, and Retrieval-augmented Generation (RAG), significantly improving document semantic understanding and information retrieval efficiency.

[0011] According to one embodiment of this application, the step of extracting semantic elements from each semantic block according to a modality-aware semantic extraction strategy includes:

[0012] Based on the content type of each semantic block, a semantic extraction strategy matching each semantic block is determined; the content types include at least tables, images, and text;

[0013] According to the semantic extraction strategy, the page content in each semantic block is extracted to obtain at least one semantic element corresponding to each semantic block; wherein, the at least one semantic element is used to update a specific field in the semantic block structure.

[0014] In the above embodiments, by introducing a modality-aware semantic extraction strategy, differentiated content extraction logic can be applied to different types of semantic block regions, avoiding mixed processing of information such as text, tables, and images. The extracted semantic elements are more consistent with the original content structure and have high semantic representativeness.

[0015] Secondly, this application provides a document processing apparatus, the apparatus comprising:

[0016] The acquisition module is used to acquire the document to be processed; the document to be processed is a portable document format.

[0017] The parsing module is used to parse the document to be processed page by page to obtain the page content of each page, and to perform semantic block recognition on the content of each page to obtain multiple semantic blocks;

[0018] The extraction module is used to extract semantic elements from each semantic block according to the modality-aware semantic extraction strategy;

[0019] The encapsulation module is used to standardize and encapsulate the semantic elements according to a predefined semantic block structure, generating multiple structure instances for association with the original content in the document to be processed in semantic tasks.

[0020] According to the document processing apparatus provided in this application, page content is obtained through page-by-page parsing, and semantic block recognition is performed on each page content to obtain multiple semantic blocks. Without introducing complex learning models, efficient identification and block segmentation of page content can be achieved based on the inherent characteristics of different content types. This avoids the problem of traditional rule templates being unable to adapt to complex page layouts, thus improving the parsing capability for multimodal mixed-format documents. Furthermore, based on a modality-aware semantic extraction strategy, semantic elements are extracted from each semantic block, improving the accuracy of semantic element extraction and avoiding the problems of semantic missingness or chaotic extraction in unstructured documents. The semantic elements are standardized and encapsulated according to a predefined semantic block structure, establishing a unified, indexable, and callable semantic data expression format. This not only enhances the structural alignment capability between different semantic types, but also preserves the semantic information and layout context of the original content in the generated structure instances, possessing high searchability and vectorization capabilities. This better supports downstream tasks such as semantic retrieval, question answering, and RAG, significantly improving document semantic understanding and information retrieval efficiency.

[0021] Thirdly, this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the document processing method as described in the first aspect above.

[0022] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the document processing method as described in the first aspect above.

[0023] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the document processing method as described in the first aspect above.

[0024] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the document processing method as described in the first aspect above.

[0025] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0026] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0027] Figure 1 This is a flowchart illustrating a document processing method provided in some embodiments of this application;

[0028] Figure 2 This is a flowchart illustrating a document processing method provided in some other embodiments of this application;

[0029] Figure 3 This is a schematic diagram illustrating the application scenario of the document processing method provided in some embodiments of this application;

[0030] Figure 4 This is a schematic diagram of the structure of the document processing apparatus provided in some embodiments of this application;

[0031] Figure 5 This is a schematic diagram of the structure of a computer device provided in some embodiments of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0033] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used in the description of this application is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms "comprising" and "having," and any variations thereof, in the description, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the description, claims, or accompanying drawings of this application are used to distinguish different objects, not to describe a specific order or hierarchy.

[0034] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0035] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "attachment" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0036] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] In this application, "multiple" refers to two or more (including two), and similarly, "multiple groups" refers to two or more (including two), and "multiple pieces" refers to two or more (including two).

[0038] Dynamic segmentation methods in related technologies typically employ a single strategy, such as semantic segmentation. This usually involves splitting PDF content into paragraphs or sentences, comparing the semantic similarity of adjacent sentences or paragraphs, and finding segmentation thresholds based on cosine distance to segment the document. However, on the one hand, segmentation strategies are simplistic, inflexible, and lack specificity: semantic segmentation is computationally and storage-intensive, and using a single segmentation strategy for the entire document cannot distinguish the inherent characteristics of different content types, leading to the mixed processing of text, tables, images, and other information, failing to provide sufficiently accurate contextual information for subsequent embedding and retrieval. On the other hand, these methods struggle to maintain the semantic association between tables, images, and the surrounding text: semantic segmentation strategies fail to preserve the semantic association between tables, images, and the surrounding text when segmenting non-textual content such as images and tables.

[0039] In view of this, embodiments of this application provide a document processing method, apparatus, computer device, storage medium, and program product. By automatically analyzing the document structure and dynamically segmenting and extracting semantics according to the content type, document segments with high semantic consistency and retrieval efficiency are generated. Portable document format documents with mixed text and image structures are converted into semantic index base data with a unified semantic structure, thereby improving the accuracy of subsequent semantic tasks.

[0040] The document processing methods provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0041] The document processing method can be applied to computer equipment, and can be executed by the hardware or software in the computer equipment.

[0042] The document processing method provided in this application can be executed by a computer device or a functional module or entity within a computer device that implements the document processing method. The following description uses a computer device as the executing entity to illustrate the document processing method provided in this application.

[0043] Figure 1 This is a flowchart illustrating a document processing method provided in some embodiments of this application. For example... Figure 1 As shown, the document processing method includes steps 110 to 140.

[0044] Step 110: Obtain the document to be processed; the document to be processed is in portable document format.

[0045] The document to be processed is an electronic document file obtained by the user through local upload, network transmission, or system reception. Documents can include, but are not limited to, portable document formats with page structure and formatted content, such as PDF, Office documents, HTML, and ePub. This embodiment uses a PDF document as an example for illustration.

[0046] For example, after receiving a document file, a computer device converts it into a portable document format to obtain a document to be processed, which serves as the input source for subsequent content analysis and semantic processing. Alternatively, the computer device may directly receive or retrieve a document to be processed in a portable document format from local storage.

[0047] Step 120: Parse the document to be processed page by page to obtain the page content of each page, and perform semantic segmentation recognition on the content of each page to obtain multiple semantic segments.

[0048] Page-by-page parsing includes identifying text objects, image objects, graphic elements, table lines, and other content components on each page of a PDF document.

[0049] Page content refers to the collection of information that makes up each page of the document to be processed, including but not limited to text objects, image objects, drawing elements (such as table lines), font styles, and position coordinates. Page content can be generated by a PDF parsing engine and used as raw input data for semantic analysis and structure extraction. For example, computer devices extract the page content of each page using PDF parsing libraries (such as PyMuPDF, PDFPlumber, etc.).

[0050] Semantic chunking recognition refers to the process by which computer devices divide a document page into several semantically independent regions based on the page's layout structure, object types, and layout information. The goal of this process is to separate the mixed-format content (such as titles, tables, images, and body text) in the original page into content blocks that are structurally independently parsable and semantically aggregated.

[0051] Correspondingly, semantic chunking refers to the content regions with type attributes and semantic boundaries that are divided from the page during the semantic chunking recognition process. Each semantic chunk represents a certain type of semantic entity in the document, such as a paragraph text block, a table, or an image, and serves as the basic processing unit for subsequent semantic element extraction and structure encapsulation.

[0052] In some embodiments, after obtaining the page content of each page, the computer device may use heuristic rules, visual flow analysis, or model-based region detection methods to divide the page into several block regions with content uniformity and semantic independence.

[0053] In other embodiments, the computer device utilizes a PDF parsing library to extract underlying components such as text blocks, image objects, and drawing line segments from the page, performing structure-driven semantic block recognition on each page's content. This outputs multiple semantic blocks, including paragraph blocks, table blocks, and image blocks, serving as the basis for subsequent semantic element extraction. Compared to methods such as visual recognition, this embodiment does not require the introduction of complex deep model semantic block recognition mechanisms, nor does it rely on model training or external labeled data, offering advantages such as simple implementation, high efficiency, and low deployment cost.

[0054] Step 130: Extract semantic elements from each semantic block according to the modality-aware semantic extraction strategy.

[0055] Modality awareness refers to the ability to automatically adjust processing strategies based on content type modality. In this application, modality awareness is reflected in the semantic element extraction process, where the computer device can select different extraction logics based on the type of semantic chunks to adapt to the differences in content structure under different modalities. The content types of semantic chunks include, but are not limited to, text, tables, and images.

[0056] Semantic extraction strategies refer to the specific processing methods used by computer devices to identify key content for different types of semantic blocks. These strategies may include rule-driven methods (such as regular expression matching, position determination, and word frequency statistics) or simple heuristics, which aim to extract text, keywords, or structural information that can represent the semantic features of each region.

[0057] Semantic elements refer to information units extracted from semantic blocks that express the main semantic content of that area. Semantic elements may include original content (such as text paragraphs, table content, image captions) and their abstract representations (such as keywords, titles, tags, etc.), and will serve as the source of input content for the structure encapsulation.

[0058] For each semantic block, the computer device automatically matches the corresponding semantic extraction strategy according to its content type. It can use differentiated detection and element extraction logic under different content modalities to extract semantic elements that represent the core meaning of the block area from each semantic block, so as to ensure that the final extracted semantic elements are representative and structurally complete.

[0059] For example, for plain text chunks, the computer device extracts keywords that can represent semantic topics; for table chunks, the computer device extracts the content of the table header and core cells that reflect the table structure and topic; for image chunks, the computer device extracts the contextual statements adjacent to the image boundaries as descriptive information, and so on.

[0060] The semantic extraction strategy does not rely on deep learning models or external corpora. Instead, it uses rule-driven methods such as structural analysis, keyword statistics, and neighboring text extraction to acquire semantic elements. Each semantic element can be used to populate subsequent structure fields and supports localization and retrieval in tasks such as semantic indexing, content retrieval, and language generation.

[0061] Step 140: Standardize and encapsulate semantic elements according to the predefined semantic chunk structure to generate multiple structure instances for use in semantic tasks to associate the original content in the document to be processed.

[0062] A semantic chunk structure is a standardized data format template used to encapsulate semantic elements. This structure typically includes uniformly defined fields, such as content, keywords, type, and section, to provide a consistent structural description for different types of semantic chunks.

[0063] For example, the semantic chunk structure is defined as a dynamic structure as follows:

[0064] {"content":…,"keywords":[key1,key2,…],"type":text / table / image,"section":…}

[0065] Standardized encapsulation refers to the process of mapping and normalizing the semantic elements extracted from each semantic block according to the field rules of the semantic block structure. The encapsulated data maintains structural consistency and has interpretability at the semantic layer, facilitating subsequent index construction and semantic application calls.

[0066] A structure instance is a specific data object generated according to the semantic chunk structure encapsulation specification, representing a structured representation of an independent semantic chunk. Each structure instance corresponds to a chunk region in a document, containing its original content, keyword information, type tags, and chapter affiliation, serving as the smallest data unit for semantic indexing and semantic application tasks.

[0067] Based on the extracted semantic elements, the computer equipment constructs a corresponding data encapsulation unit for each semantic block and standardizes the encapsulation according to a predefined semantic block structure template. This structure template is uniformly applicable to different types of content areas. By defining common fields such as content fields, keyword fields, type fields, and chapter fields, it achieves structural alignment of multimodal block results such as tables, images, and text. This encapsulation process ensures that the semantic content has a consistent data format, facilitating subsequent index building, storage management, and semantic task invocation.

[0068] During the encapsulation process, the computer device can also record the page number, order of appearance, and other positional information of each structure instance in the original document to ensure that the structure sequence remains consistent with the original document structure. The final set of structure instances can serve as the basic data unit for semantic indexing, supporting semantic application tasks such as vector retrieval and content generation.

[0069] Semantic tasks refer to information processing operations that use structured instances as the basic data source and are completed through vector retrieval, content matching, language generation, and other methods. Semantic tasks include, but are not limited to, semantic retrieval, question answering, summary generation, and retrieval enhancement, aiming to support more accurate content understanding and retrieval based on structured semantic representation.

[0070] The document processing method provided in this application obtains page content through page-by-page parsing and performs semantic block recognition on each page content to obtain multiple semantic blocks. Without introducing complex learning models, it can efficiently identify and block page content based on the inherent characteristics of different content types. This avoids the problem of traditional rule templates being unable to adapt to complex page layouts, thus improving the parsing capability for multimodal mixed-format documents. Furthermore, based on a modality-aware semantic extraction strategy, semantic elements are extracted from each semantic block, improving the accuracy of semantic element extraction and avoiding the problems of semantic missing or chaotic extraction in unstructured documents. The semantic elements are standardized and encapsulated according to a predefined semantic block structure, establishing a unified, indexable, and callable semantic data expression format. This not only enhances the structural alignment capability between different semantic types but also preserves the semantic information and layout context of the original content in the generated structure instances, possessing high searchability and vectorization capabilities. This better supports downstream tasks such as semantic retrieval, question answering, and RAG, significantly improving document semantic understanding and information retrieval efficiency.

[0071] In related technologies, semantic information extraction often employs uniform, fixed rules or models, failing to differentiate between content types based on their structural and semantic variations. Because text, tables, and images differ significantly in their semantic representation, a uniform extraction method often fails to fully reflect the semantic features of each content type, resulting in incomplete or distorted information extraction and affecting the accuracy of subsequent structural representation and task invocation.

[0072] Therefore, in some embodiments, semantic elements are extracted from each semantic block according to a modality-aware semantic extraction strategy, including: determining a semantic extraction strategy that matches each semantic block based on the content type of each semantic block; the content type includes at least tables, images, and text; extracting the page content in each semantic block according to the semantic extraction strategy to obtain at least one semantic element corresponding to each semantic block; wherein, at least one semantic element is used to update a specific field in the semantic block structure.

[0073] After completing semantic block recognition, the computer device assigns a content type identifier to each block area. For example, "text" represents text, "table" represents a table, and "image" represents an image. For instance, if the computer device detects a table area on a page, it will assign the content type "table" accordingly.

[0074] Based on the identification of the content type of each semantic block, the computer device selects its own matching semantic extraction strategy when processing each semantic block to obtain its respective semantic elements. For example, table blocks use extraction logic based on structural lines and cells, image blocks use extraction methods that combine image boundaries with adjacent text, and text regions extract keywords through word frequency statistics, etc.

[0075] The extracted semantic elements are not used directly in the output. Instead, they are mapped and populated into the corresponding positions in the structure according to the field requirements of the structure template, such as content fields, keyword fields, and chapter fields. In this way, different types of content have a unified semantic dimension during the structure encapsulation stage.

[0076] The semantic block structure includes at least a type field, a chapter field, a content field, and a keyword field. The type field identifies the semantic block type, such as text, table, or image. The chapter field identifies the chapter to which the semantic block belongs. The content field stores the core text extracted from the semantic block. The keyword field stores the set of keywords obtained by the semantic extraction strategy.

[0077] In the above embodiments, by introducing a modality-aware semantic extraction strategy, differentiated content extraction logic can be applied to different types of semantic block regions, avoiding mixed processing of information such as text, tables, and images. The extracted semantic elements are more consistent with the original content structure and have high semantic representativeness.

[0078] Furthermore, the segmentation results in related technologies lack detailed extraction and mapping of key elements within semantic units, making it difficult to directly support tasks such as semantic indexing and generation. This results in incomplete semantic understanding dimensions and imprecise index granularity, affecting the efficient utilization of document structure and content.

[0079] Therefore, in some embodiments, semantic elements include at least chapter elements, content elements, and keyword elements; according to a semantic extraction strategy matching each semantic block, at least one semantic element is extracted from each semantic block, including: identifying title text lines with title features in each semantic block to obtain chapter elements, so as to update the chapter field of the semantic block structure corresponding to each semantic block; for any semantic block of table, image, and text content types, content elements and keyword elements are extracted according to a semantic extraction strategy matching the content type of the targeted semantic block; wherein, content elements are used to update the content field of the semantic block structure, and keyword elements are used to update the keyword field of the semantic block structure.

[0080] During the semantic extraction stage, the computer equipment not only distinguishes content types but also further divides and extracts semantic elements for each semantic block, including at least three dimensions: chapter elements, content elements, and keyword elements. The introduction of chapter elements gives each structure a structural belonging relationship, supporting contextual positioning and chapter navigation; the field mapping between content elements and keyword elements gives the structure high searchability and context recovery capability.

[0081] The chapter element is determined by the computer device during page parsing by identifying text lines with title characteristics (such as bold font, large font size, regular expression matching "Chapter x"); this title information forms a belonging relationship with the subsequent content blocks, and is then filled into the chapter field of each structure. For example, on a certain page, if the computer device identifies the title "Chapter Two Principles" as a chapter element, the content after this title is uniformly marked as belonging to that chapter.

[0082] For specific content blocks, such as tables, images, and body paragraphs, the computer device selects the corresponding semantic extraction strategy based on their type to extract content elements and keyword elements. Content elements include, for example, body paragraph text, adjacent descriptive text for images, and structured text for tables; keyword elements include, for example, body keyword text, text in the context of images, and the first row of text in tables. After extraction, the computer device fills each semantic element into the corresponding fields of the structure according to their semantic meaning, such as the type field, content field, keywords field, and section field, achieving fine-grained filling of the semantic dimensions of the structure.

[0083] For example, if a computer device extracts a section of text and uses word frequency statistics to identify high-frequency words such as "voltage," "current," and "Ohm's Law" as keywords, then the computer device will fill the entire text as content elements into the content field and fill the keywords into the keyword field.

[0084] In the above embodiments, by introducing semantic element-level extraction and field mapping in the structure generation stage, a more granular and semantic structural representation of document content can be achieved. This representation has high scalability, clear hierarchy, and semantic integrity, and can significantly improve the accuracy and response quality of document expression in tasks such as semantic indexing, vector retrieval, and language generation.

[0085] Furthermore, each semantic extraction strategy is implemented through a rule-driven approach, eliminating the need for deep learning models or external corpora. This effectively reduces computational and storage resource consumption, allows for flexible deployment and efficient operation, and can adapt to various mixed-format document structures. It also contributes to the accuracy and semantic consistency of semantic indexing, semantic retrieval, and generation tasks.

[0086] In related technologies, structured document processing methods often neglect the identification and allocation of chapter titles, resulting in a lack of clear semantic attribution tags for subsequent content and affecting the contextual association capabilities of semantic indexing. Especially in unstructured documents with mixed text and images, title text is not easily bound to the body text, causing logical disconnects between structural blocks.

[0087] Therefore, in some embodiments, identifying title text lines with title features in each semantic block to obtain chapter elements includes: identifying title text lines with title features based on the layout structure in the page content; title features include at least one of indentation level, font size difference, or regular expression matching results; extracting the text content in the identified title text lines to obtain chapter elements; wherein, when multiple title texts are included in the same page, the first title text line is taken as a chapter element.

[0088] During the page-by-page parsing process, computer equipment identifies text lines with chapter title characteristics by detecting the text layout structure on the page. Title characteristics may include, but are not limited to, one or more of the following: font size significantly larger than the average body text font, paragraphs without indentation or fully left-aligned, short text lines centered, or pattern matching results that satisfy regular expression rules (such as "Chapter x", "Section x", etc.).

[0089] Once a text line matching the title characteristics is detected, the computer device extracts it as a chapter element and establishes a chapter affiliation relationship with each subsequent semantic block, thereby marking the chapter in the chapter field of the structure.

[0090] When multiple heading text lines are detected on the same page, the computer device can select the first heading line closest to the top of the page as the chapter element of the current page based on the line position priority rule, so as to ensure the stability and consistency of chapter identification.

[0091] When a chapter change is detected, i.e. a new title is detected, the computer device ends the current semantic block and updates the section field, and then proceeds to detect and extract the next semantic block.

[0092] In the above embodiments, by introducing a chapter element recognition mechanism, the logical chapter hierarchy of a document can be accurately extracted and maintained, improving the semantic attribution and contextual coherence between structures. Compared to structured results without chapter identifiers, structures with chapter fields can support stronger navigation, clustering, and paragraph-level query capabilities, significantly enhancing the semantic retrieval system's expressive ability in terms of document structure awareness and cross-segment association.

[0093] In formats such as PDF, table boundary information is mostly represented by drawing instructions, lacking table models or metadata support. This makes it difficult for traditional text extraction methods to recover the original structure and semantic units of the table, resulting in information distortion and limiting the usability of table data in indexing, matching, and generation tasks.

[0094] Therefore, in some embodiments, when the semantic block is of table type, for any semantic block of table, image, and text content type, content elements and keyword elements are extracted according to a semantic extraction strategy that matches the content type of the semantic block. This includes: analyzing the drawing items in the page content to detect wireframe structures containing several horizontal and vertical lines to identify table boundary areas; extracting the text content within the table boundary areas and structuring the text content according to rows and columns to obtain structured text, and using the structured text as content elements associated with the table; identifying the table header text within the table boundary areas and using the table header text as keyword elements associated with the table.

[0095] When a semantic block region is identified as a table type, the computer device parses drawing instructions (such as line and rect objects) on the page to detect enclosed regions formed by horizontal and vertical lines. When a certain number and arrangement of wireframe structures are detected, the computer device determines that the region is a table boundary region. For example, if at least three horizontal lines and two vertical lines are detected, a table region is determined to have been detected.

[0096] For example, the computer device creates an empty list `tables` to store the detected tables, and iterates through the drawing items on the page, identifying horizontal and vertical lines and storing them in the lists `horizontal_lines` and `vertical_lines`, respectively. Based on all detected lines, the computer device calculates the minimum and maximum boundaries of the tables, including the leftmost boundary `min_x`, the top boundary `min_y`, the rightmost boundary `max_x`, and the bottom boundary `max_y` of the table area, which together form the rectangular area of ​​the table.

[0097] After identifying the table area, the computer device extracts the text content, for example, by extracting the text content of the table area according to the boundaries calculated by fitz.Rect(min_x,max_x,min_y,max_y). The computer device performs structured processing on the text according to the row and column coordinate information, such as dividing it into rows by horizontal lines and columns by vertical lines, thereby constructing a two-dimensional structured text representation, which is then filled into the content fields of the structure as content elements.

[0098] At the same time, the computer device recognizes the field name text in the first row or column of the table, such as "name", "value", "unit", etc., and uses these semantically identifying texts as keyword elements to fill the keyword field of the structure to represent the subject characteristics of the table.

[0099] For example, if the table content is valid (i.e., there is at least one row and at least two cells in each row), the computer device adds it to the table list. The table format is table:List[List[str]].

[0100] In the above embodiments, by introducing wireframe detection and structured processing mechanisms into the semantic blocks of the table, the two-dimensional structural expression of the table can be quickly and accurately restored without relying on complex visual models. This avoids the information loss caused by flattening the table into a continuous text without hierarchy. At the same time, by using the field names in the first row or the content of key cells to form keyword elements, the semantic representation ability and retrieval accuracy of the table structure are improved.

[0101] Image content is often difficult to identify. Related technologies usually use visual recognition technology to identify images, but this method requires a lot of computing resources, and its effectiveness depends on the recognition ability of the visual recognition model.

[0102] Therefore, in some embodiments, when the semantic block is of the image type, for any semantic block of table, image, and text content type, content elements and keyword elements are extracted according to a semantic extraction strategy that matches the content type of the semantic block, including: determining the boundary position of the image on the page by extracting embedded objects in the page content; extracting contextual text in the text lines adjacent to the boundary position information and using the contextual text as the content element associated with the image; and extracting keywords in the contextual text and using the keywords as the keyword element associated with the image.

[0103] When a semantic block is identified as an image, the computer device uses a PDF parsing tool to extract the embedded image objects from the page and records their boundary position information (such as x, y coordinates and width and height) on the page. For example, the computer device uses `page.get_images(full=True)` to get all image objects on the current page, iterates through each image, extracts the image and obtains its position information on the page using `page.get_image_rects()`, and finally adds the image's bounding box information to a list and returns it.

[0104] The computer device uses this boundary location as the center and queries text lines within a certain distance (e.g., 3-5 lines before and after) in the context direction, identifying text with descriptive or annotative features as the contextual text content associated with the image. This text is summarized as content elements and used to populate the content fields of the structure.

[0105] For example, computer devices first use regular expressions to match common image title formats (such as "..."). Figure 1 If a matching title is found, the title is returned directly and filled into the chapter field of the structure as a chapter element. If no title is found, the computer device estimates the approximate position of the image in the text based on the bounding box (bbox) information of the image. Assuming the page height is 1000, the position of the image in the text is mapped by calculating the ratio of the vertical position (bbox[1]) of the image to the page height. Then, based on the estimated position, the computer device finds the corresponding line in the text, extracts the content of the line and the lines before and after it as context text, and maps the extracted context content to the content field of the structure.

[0106] Furthermore, the computer device can perform keyword extraction processing on the context text, such as removing stop words, counting word frequencies, and filtering noun phrases, to extract a representative set of keywords and fill them into the keyword field of the structure, thereby enhancing the semantic searchability of the image block.

[0107] In the above embodiments, by using boundary localization and neighboring text extraction, an effective semantic association can be established between the image and its context. This enables accurate restoration of the information meaning expressed by the image in structured semantic processing, significantly enhancing the participation of image regions in semantic indexing, image-text understanding, and RAG tasks. It provides a more accurate image-text correspondence basis for semantic retrieval and generation tasks, and enhances the expressive power and calling effect of image-related content in question-answering systems, semantic retrieval, and other scenarios.

[0108] In related technologies, the processing of ordinary text segments usually focuses on whole-block extraction, lacking effective modeling of semantic features within paragraphs. Especially in keyword extraction, it often relies on training models or dictionary matching, which is difficult to adapt to documents with complex structures or non-standard language. This results in the semantic fragments expressed by the structure being lengthy, unfocused, and missing keywords, affecting indexing efficiency and recall quality.

[0109] Therefore, in some embodiments, when the semantic block is text type, for any semantic block of table, image, and text content type, content elements and keyword elements are extracted according to a semantic extraction strategy that matches the content type of the semantic block. This includes: dividing continuous text content into multiple paragraphs and filtering them to obtain multiple non-empty paragraphs; accumulating the paragraph content of multiple non-empty paragraphs and using the accumulated paragraph content as content elements; performing word frequency analysis on the accumulated paragraph content, selecting the multiple words with the highest frequency based on the word frequency analysis results, and using the multiple words as keyword elements; wherein, if the text length of the current semantic block exceeds a character threshold, the current semantic block is terminated, and the next semantic block is processed.

[0110] When a semantically segmented region is identified as text, the computer device first performs preliminary cleaning and segmentation of its original text content. Specifically, the computer device can divide continuous text into several non-empty paragraphs based on rules such as line breaks, paragraph blanks, and punctuation breaks, and filter out empty paragraphs or segments containing only invalid symbols. For example, the computer device can use regular expressions such as r"\n\s*\n" to split the text into paragraphs, remove leading and trailing whitespace characters from each paragraph, and finally return a list of non-empty paragraphs.

[0111] The computer equipment accumulates and concatenates these non-empty paragraphs to form a complete semantic expression unit, which serves as the content element of the semantic block and is then filled into the content field of the structure. This content element represents the overall semantic content of the current text block and possesses the integrity of the original information.

[0112] Next, the computer performs word frequency statistical analysis on the accumulated content, sorts the most frequently occurring words, and selects the top N high-frequency words as the keyword elements of that text block. Keywords are used to reflect the main idea and semantic features of the text and are filled into the keyword field of the structure.

[0113] To prevent a single structure from becoming too long, computer devices can set a character threshold (such as 1500 characters). When the cumulative content length exceeds this threshold, the current semantic block ends and a new semantic block region is opened, ensuring that the granularity of the structure is controllable.

[0114] In the above embodiments, by introducing lightweight strategies such as paragraph segmentation, content accumulation and word frequency extraction, an automatic keyword extraction process without model training is realized. This not only improves the semantic concentration and retrieval accuracy of the structure expression, but also avoids the structural redundancy problem caused by excessively long text blocks. This makes the document structure results more refined and the semantics more focused, which is convenient for efficient retrieval and accurate matching in subsequent tasks such as semantic retrieval and question answering.

[0115] While related technologies can identify text, images, or table content, the lack of a unified data structure to encapsulate their semantic elements leads to inconsistent structures and field definitions, making it difficult to support subsequent semantic indexing and task invocation. Particularly in multimodal documents, the differences in format and semantics between different semantic blocks make it difficult for subsequent retrieval systems to align and process them, affecting information reconstruction and consistency of expression.

[0116] Therefore, in some embodiments, semantic elements are mapped to field types of semantic block structures, with field types including at least type fields, chapter fields, content fields, and keyword fields. Semantic elements are standardized and encapsulated according to a predefined semantic block structure to generate multiple structure instances. This includes: creating a structure instance for each semantic block, with each instance containing field variables for type, chapter, content, and keyword fields; assigning values ​​to the type variables in the corresponding structure instance based on the content type of each semantic block; and assigning the chapter, content, and keyword elements corresponding to each semantic block to the chapter, content, and keyword variables in their respective structure instances to complete the standardized encapsulation and generate multiple structure instances.

[0117] The computer device creates a corresponding structure instance for each semantic block region in the document based on a predefined semantic block structure template. This structure instance includes variable fields such as type, chapter, content, and keywords, according to field specifications.

[0118] Accordingly, the computer device fills in the corresponding structure type field based on the content type of the semantic block determined during the segmentation process, assigning values ​​to the type field. Based on the chapter to which the semantic block belongs, the computer device assigns values ​​to the chapter field using chapter elements. Based on the content elements and keyword elements extracted by the semantic extraction strategy, the computer device fills in the corresponding fields of the structure, assigning values ​​to the content field and keyword field, respectively.

[0119] Ultimately, each semantic block is encapsulated into a set of structured instances with a unified structure and clear semantics, possessing standardized structured semantic expression capabilities.

[0120] In the above embodiments, by establishing a unified semantic block structure and standardizing field mapping and encapsulation of different types of semantic elements, the expressive standardization and reusability of document structuring results are significantly improved; each structure has consistent field dimensions, which is conducive to building a unified index system, and field-level expression enables semantic information to be accurately located, organized and called, improving the accuracy of results for tasks such as semantic retrieval, vector matching, and language generation.

[0121] In some embodiments, the document processing method provided in this application further includes: loading a semantic vector model, and performing vectorization operations on each structure instance through the semantic vector model to obtain multiple vectorized representations; storing the multiple vectorized representations in sequence in a vector library for subsequent semantic tasks to call; wherein the order of the multiple vectorized representations is consistent with the order of each semantic block corresponding to the corresponding structure instance in the document to be processed.

[0122] After completing semantic element extraction and structure encapsulation, the computer device loads a pre-trained semantic vector model (such as BERT, bge, text-embedding series, etc.) and performs vectorization operations on the content fields in the structure instance.

[0123] Vectorization operations can include: using the content fields in the structure as semantic encoding input for semantic encoding, and generating a corresponding high-dimensional semantic vector representation from the encoding result.

[0124] Following the order in which the structures appear in the original document, the computer device encodes each structure one by one, and the resulting multiple semantic vectors are written into a semantic vector library in the same order. This order corresponds to the natural arrangement of semantic chunks in the document, ensuring that the content context and semantic consistency are maintained in subsequent semantic tasks.

[0125] In the above embodiments, by performing semantic vectorization encoding on the structure instances and writing them sequentially into the vector library according to their order in the original document, it is possible to ensure fine-grained semantic representation, support block-level semantic retrieval, and preserve the original context order, thereby improving the rationality of semantic content organization and the accuracy of invocation, and providing a more stable and structured representation foundation for semantic application tasks.

[0126] Figure 2 This is a flowchart illustrating a document processing method provided in other embodiments of this application. For example... Figure 2 As shown, the document processing method provided in this application adopts a dynamic chunking strategy and defines a semantic chunking structure. The computer device first extracts the document to be processed as input based on the PDF document path, and then parses the document based on PDF parsing tools such as PyMuPDF (fitz), obtaining the page content of each page through pagination processing.

[0127] For each page's content, the computer device detects chapters, tables, images, and text. In chapter title detection, the computer device uses regular expressions to identify chapter titles; when a chapter change is detected, the current text block ends and the chapter information is updated. Specifically, this includes:

[0128] a) Based on the parsed content of the current page, check each line according to the regular expression to see if it is an article title. If it is, store the title in the headers list and return it.

[0129] b) When a page has multiple titles, only the content of the first title is displayed.

[0130] c) When a chapter change is detected, i.e., when a new title appears, the current text block ends and the chapter information section is updated.

[0131] In table detection and processing, computer equipment uses page drawing information (detecting horizontal and vertical lines) to identify table areas, formats the table content into text, and extracts table headers and row titles as keywords. Specifically, this includes:

[0132] a) Enter the content of the current page.

[0133] b) Initialization: Create an empty list named tables to store the detected tables.

[0134] c) Line segment detection: Traverse the drawing items on the page, identify horizontal and vertical lines, and store them separately.

[0135] The lists of horizontal_lines and vertical_lines.

[0136] d) Determine if it is a table: If at least 3 horizontal lines and 2 vertical lines are detected, a table area is considered to exist.

[0137] e) Calculate table boundaries: Based on all detected lines, calculate the minimum and maximum boundaries of the table (min_x, max_x, min_y, max_y). Where: min_x: the leftmost boundary of the table region (minimum x-coordinate); min_y: the top boundary of the table region (minimum y-coordinate); max_x: the rightmost boundary of the table region (maximum x-coordinate); max_y: the bottom boundary of the table region (maximum y-coordinate), used to determine the rectangular area of ​​the table.

[0138] f) Extract table text: Use the boundaries calculated by fitz.Rect(min_x,max_x,min_y,max_y) to extract the text content of the table region.

[0139] g) Structured table content: The extracted text is split into rows and further split into cells to form a two-dimensional list table.

[0140] h) Store tables: If the table content is valid (i.e., there is at least one row and each row has at least two cells), add it to the list of tables. The format of the table is table:List[List[str]]. For example: [[["Heading 1","

[0141] [Heading 2","Heading 3"],["Content 1","Content 2","Content 3"],["Content 4","Content 5","Content 6"]],[["Column 1","Column 2"],["Value 1","Value 2"],["Value 3","Value 4"]]]

[0142] i) Iterate through `tables` and update the content of each block. Extract the table text content and assign it to the `content` variable of the block. Format the above two-dimensional list into table text separated by vertical bars (|). The return value is a string, where each line represents a row of the table, and columns are separated by "|". Extract the keywords from the table content and assign them to the `keywords` variable of the block. Extract the first row and first column of the table as the keyword content for the table block. Assign the `type` value to `table` for the block, and assign the `section` value based on the title detection results.

[0143] In image detection and processing, computer devices identify images on a webpage, extract their location within the page, and generate descriptive information based on the surrounding text. Specifically, this includes:

[0144] a) Enter the content of the current page.

[0145] b) Get all image objects on the current page using page.get_images(full=True).

[0146] c) Iterate through each image, extract the image and obtain its position information on the page using the page.get_image_rects() method, and finally add the image's bounding box information to a list and return it.

[0147] d) Traverse the map boundary information and extract the text surrounding the image as context:

[0148] i. Find image titles: Use regular expressions to match common image title formats (such as "..."). Figure 1 "、

[0149] (e.g., “Figure 1”). If a matching title is found, return that title directly.

[0150] ii. Estimating Image Position: If the title is not found, estimate the approximate position of the image within the text based on the image's bounding box (bbox) information. Assuming a page height of 1000, calculate the image's vertical position.

[0151] The ratio of (bbox[1]) to the page height is mapped to the position in the text.

[0152] iii. Extracting Context: Based on the estimated position, locate the corresponding line in the text and extract the content of that line and the lines before and after it as context. Finally, return the filtered context text.

[0153] iv. Assign the extracted context content to the content variable of the block.

[0154] v. Extracting Image Keywords: Extract keywords from the context and assign them to the `keywords` variable in the block. First, stop words and punctuation are removed, then a simple word frequency count is performed, and finally, the top 10 most frequent words are selected as keywords and returned.

[0155] vi. Assign the type value of the block to image.

[0156] vii. The section of the block is assigned a value based on the result of the title detection.

[0157] In text segmentation processing, computer devices process text segments on the current page, excluding titles, tables, and images. Specifically, this includes:

[0158] a) Dynamic keyword extraction: High-frequency keywords are extracted based on the text content within each block using a simple word frequency statistics method.

[0159] b) Text block processing: The text content is accumulated by paragraph, and a text block is generated when a preset threshold is reached, ensuring that each block is of appropriate length and semantically complete.

[0160] c) Add type tags: For each generated block, add the corresponding type tags (such as [text], [table], [image]) and keywords.

[0161] Finally, the computer device obtains the results of each block, loads the vector model, embeds the content of each block, and stores it in the Chroma vector library.

[0162] In some embodiments, semantic tasks include at least retrieval tasks and generation tasks. The document processing method provided in this application further includes: receiving query information containing a user's semantic intent, and vectorizing the query information to obtain a query vector; retrieving a matching target structure instance from a vector library based on the query vector; and determining the original content in the document to be processed that matches the query information based on the target structure instance; wherein the original content is used as the output of the retrieval task and / or as auxiliary input for the generation task.

[0163] Computer devices can support a variety of semantic tasks based on the built vector library. Semantic tasks include retrieval tasks and generation tasks. Retrieval tasks include semantic query, paragraph location, document matching, keyword recall, etc., while generation tasks include question answering, summary generation, retrieval augmentation (RAG), etc.

[0164] Upon receiving a query from a user, the computer first performs semantic vectorization to generate a corresponding query vector. Then, it performs similarity matching in a vector database, retrieving the structure instance with the highest similarity to the query vector, which is then used as the target structure instance. This process can be based on a TopK search using vector distance (such as cosine similarity) to obtain one or more target structure instances.

[0165] Furthermore, the computer device can locate the content range in the original document based on one or more of the content fields, keyword fields, and chapter fields in the target structure, and use it as the direct output of retrieval tasks, such as paragraph location or summary display, or as the semantic context input for generation tasks, to prompt the large language model to generate the answer or content fragment expected by the user.

[0166] Figure 3 This is a schematic diagram illustrating application scenarios of the document processing method provided in some embodiments of this application. For example... Figure 3 As shown, users can enter query information on a computer device to ask questions or instruct the generation of specific content. For example, a user might enter the query: "What operational strategies are mentioned in this document?". The computer device encodes the query as a vector Q and retrieves the closest structure instance from the structure examples 1 to structure instances N stored in the vector library, such as:

[0167] Structure instance x: {"content": Operation strategies include..., "keywords": [strategy 1, strategy 2, ...], "type: text, "section": Chapter 3 Introduction to Operation Strategies}

[0168] The computer device can then return the content of the structure as a search result, or locate the original text position and return the original text paragraph by giving the structure instance, or concatenate its content with related structures and input it into the generative model to generate a natural language answer: "The operational strategies mentioned in this document include strategy 1, strategy 2, etc."

[0169] In the above embodiments, the semantic vector retrieval based on structure instances and the original text content association mechanism achieves precise docking between structured semantic representation and semantic tasks, improves the recall accuracy of semantic retrieval tasks, and improves the answer quality of generation tasks.

[0170] The document processing method provided in this application can be executed by a document processing device. This application uses a document processing device executing the document processing method as an example to illustrate the document processing device provided in this application.

[0171] This application also provides a document processing apparatus applied to computer equipment.

[0172] Figure 4 This is a schematic diagram of the structure of a document processing apparatus provided in some embodiments of this application. For example... Figure 4 As shown, the document processing device includes an acquisition module 401, a parsing module 402, an extraction module 403, and a packaging module 404. Wherein:

[0173] The acquisition module 401 is used to acquire the document to be processed; the document to be processed is a portable document format.

[0174] The parsing module 402 is used to parse the document to be processed page by page to obtain the page content of each page, and to perform semantic block recognition on the content of each page to obtain multiple semantic blocks.

[0175] Extraction module 403 is used to extract semantic elements from each semantic block according to the modality-aware semantic extraction strategy.

[0176] The encapsulation module 404 is used to standardize and encapsulate semantic elements according to a predefined semantic chunk structure, generating multiple structure instances for association with the original content in the document to be processed in semantic tasks.

[0177] The document processing apparatus provided in this application obtains page content through page-by-page parsing and performs semantic block recognition on each page content to obtain multiple semantic blocks. Without introducing complex learning models, it can efficiently identify and block page content based on the inherent characteristics of different content types. This avoids the problem of traditional rule templates being unable to adapt to complex page layouts, thus improving the parsing capability for multimodal mixed-format documents. Furthermore, based on a modality-aware semantic extraction strategy, semantic elements are extracted from each semantic block, improving the accuracy of semantic element extraction and avoiding semantic loss or chaotic extraction problems in unstructured documents. The semantic elements are standardized and encapsulated according to a predefined semantic block structure, establishing a unified, indexable, and callable semantic data expression format. This not only enhances the structural alignment capability between different semantic types but also preserves the semantic information and layout context of the original content in the generated structure instances, possessing high searchability and vectorization capabilities. This better supports downstream tasks such as semantic retrieval, question answering, and RAG, significantly improving document semantic understanding and information retrieval efficiency.

[0178] In some embodiments, the extraction module is further configured to determine a semantic extraction strategy that matches each semantic block based on the content type of each semantic block; the content type includes at least tables, images and text; and extract the page content in each semantic block according to the semantic extraction strategy to obtain at least one semantic element corresponding to each semantic block; wherein, at least one semantic element is used to update a specific field in the semantic block structure.

[0179] In some embodiments, semantic elements include at least chapter elements, content elements, and keyword elements; the extraction module is also used to identify title text lines with title features in each semantic block to obtain chapter elements, so as to update the chapter field of the semantic block structure corresponding to each semantic block; for any semantic block of table, image, and text content types, content elements and keyword elements are extracted respectively according to the semantic extraction strategy matching the content type of the targeted semantic block; wherein, the content elements are used to update the content field of the semantic block structure, and the keyword elements are used to update the keyword field of the semantic block structure.

[0180] In some embodiments, the extraction module is further configured to identify title text lines with title features based on the layout structure in the page content; the title features include at least one of indentation level, font size difference, or regular expression matching result; extract the text content in the identified title text lines to obtain chapter elements; wherein, in the case of multiple title texts on the same page, the first title text line is taken as a chapter element.

[0181] In some embodiments, when the semantic block is a table type, the extraction module is further configured to identify the table boundary area by analyzing the drawing items in the page content, detecting the wireframe structure containing several horizontal and vertical lines; extracting the text content within the table boundary area, and structuring the text content according to rows and columns to obtain structured text, and using the structured text as the content element associated with the table; identifying the table header text within the table boundary area, and using the table header text as the keyword element associated with the table.

[0182] In some embodiments, when the semantic block is an image type, the extraction module is further configured to determine the boundary position of the image on the page by extracting embedded objects in the page content; extract context text from the text lines adjacent to the boundary position information and use the context text as content elements associated with the image; extract keywords from the context text and use the keywords as keyword elements associated with the image.

[0183] In some embodiments, when the semantic block is text type, the extraction module is further configured to divide the continuous text content into multiple paragraphs and perform filtering to obtain multiple non-empty paragraphs; accumulate the paragraph content of multiple non-empty paragraphs and use the accumulated paragraph content as content elements; perform word frequency analysis on the accumulated paragraph content, select the multiple words with the highest frequency according to the word frequency analysis results, and use the multiple words as keyword elements; wherein, if the text length of the current semantic block exceeds the character threshold, the current semantic block is terminated and the next semantic block is processed.

[0184] In some embodiments, semantic elements are mapped to the field types of semantic block structures, and the field types include at least type fields, chapter fields, content fields, and keyword fields. The encapsulation module is also used to create structure instances for each semantic block, and the structure instances include field variables for type fields, chapter fields, content fields, and keyword fields. According to the content type of each semantic block, the type variables in the corresponding structure instances are assigned values. The chapter elements, content elements, and keyword elements corresponding to each semantic block are assigned values ​​to the chapter variables, content variables, and keyword variables in their respective structure instances to complete standardized encapsulation and generate multiple structure instances.

[0185] In some embodiments, the above-described apparatus further includes a storage module for loading a semantic vector model and performing vectorization operations on each structure instance through the semantic vector model to obtain multiple vectorized representations; storing the multiple vectorized representations sequentially in a vector library for subsequent semantic tasks to call; wherein the order of the multiple vectorized representations is consistent with the order of the semantic blocks corresponding to the respective structure instances in the document to be processed.

[0186] In some embodiments, the semantic task includes at least a retrieval task and a generation task; the apparatus further includes an application module for receiving query information containing a user's semantic intent, and vectorizing the query information to obtain a query vector; retrieving a matching target structure instance from a vector library based on the query vector; and determining the original content in the document to be processed that matches the query information based on the target structure instance; wherein the original content is used as the output of the retrieval task and / or as an auxiliary input of the generation task.

[0187] The document processing device in this application embodiment can be a computer device or a component within a computer device, such as an integrated circuit or a chip. The computer device can be a terminal device or a server. For example, the computer device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle computer device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0188] The document processing device in this application embodiment can be a device with an operating system. The operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0189] The document processing apparatus provided in this application embodiment can implement the various processes implemented in each method embodiment, and will not be described again here to avoid repetition.

[0190] Figure 5 This is a schematic diagram of the structure of a computer device provided in some embodiments of this application. In some embodiments, such as Figure 5 As shown, this application embodiment also provides a computer device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, it implements the various processes of the above-described method embodiments and can achieve the same technical effects. To avoid repetition, it will not be described again here.

[0191] It should be noted that the computer devices in this application embodiment include the mobile computer devices and non-mobile computer devices described above.

[0192] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described document processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0193] The processor is the processor in the computer device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0194] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described document processing method.

[0195] The processor is the processor in the computer device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0196] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described document processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0197] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0198] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0199] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0200] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0201] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0202] Unless otherwise specified, all embodiments and optional embodiments of this application can be combined to form new technical solutions.

[0203] Unless otherwise specified, all technical features and optional technical features of this application may be combined to form new technical solutions.

[0204] Unless otherwise specified, all steps of this application may be performed sequentially or randomly, preferably sequentially. For example, the method includes steps (a) and (b), indicating that the method may include steps (a) and (b) performed sequentially, or it may include steps (b) and (a) performed sequentially. For example, the mention that the method may also include step (c) indicates that step (c) may be added to the method in any order; for example, the method may include steps (a), (b), and (c), or it may include steps (a), (c), and (b), or it may include steps (c), (a), and (b), etc.

[0205] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A document processing method, characterized in that, The method includes: Obtain the document to be processed; the document to be processed is in a portable document format. The document to be processed is parsed page by page to obtain the page content of each page, and semantic block recognition is performed on the content of each page to obtain multiple semantic blocks; Based on the modality-aware semantic extraction strategy, semantic elements are extracted from each semantic block. According to the predefined semantic chunk structure, the semantic elements are standardized and encapsulated to generate multiple structure instances, which are used to associate the original content in the document to be processed in the semantic task.

2. The document processing method according to claim 1, characterized in that, The step of extracting semantic elements from each semantic block according to the modality-aware semantic extraction strategy includes: Based on the content type of each semantic block, a semantic extraction strategy matching each semantic block is determined; the content types include at least tables, images, and text; According to the semantic extraction strategy, the page content in each semantic block is extracted to obtain at least one semantic element corresponding to each semantic block; wherein, the at least one semantic element is used to update a specific field in the semantic block structure.

3. The document processing method according to claim 2, characterized in that, The semantic elements include at least chapter elements, content elements, and keyword elements; the extraction of at least one semantic element from each semantic block according to a semantic extraction strategy matching each semantic block includes: Identify the title text lines with title features in each semantic block to obtain the chapter elements, and update the chapter field of the semantic block structure corresponding to each semantic block; For any semantic block of table, image, and text content types, extract content elements and keyword elements according to the semantic extraction strategy that matches the content type of the semantic block. The content element is used to update the content field of the semantic block structure, and the keyword element is used to update the keyword field of the semantic block structure.

4. The document processing method according to claim 3, characterized in that, The process of identifying title text lines with title features in each semantic block to obtain chapter elements includes: Based on the layout structure of the page content, identify title text lines with title features; the title features include at least one of indentation level, font size difference, or regular expression matching results; Extract the text content from the identified title text lines to obtain the chapter elements; where there are multiple title texts on the same page, the first title text line is taken as the chapter element.

5. The document processing method according to claim 3, characterized in that, When the semantic block is of table type, for any semantic block of table, image, and text content type, content elements and keyword elements are extracted according to the semantic extraction strategy that matches the content type of the semantic block, including: By analyzing the drawing items in the page content, we can detect wireframe structures containing several horizontal and vertical lines to identify table boundary areas. Extract the text content within the boundary area of ​​the table, and perform structuring processing on the text content according to rows and columns to obtain structured text, and use the structured text as content elements associated with the table; Identify the header text within the bounded area of ​​the table and use the header text as a keyword element associated with the table.

6. The document processing method according to claim 3, characterized in that, When the semantic block is an image type, for any semantic block of table, image, or text content type, content elements and keyword elements are extracted according to a semantic extraction strategy that matches the content type of the semantic block, including: The image's boundary position on the page is determined by extracting embedded objects from the page content; Extract contextual text from the text lines adjacent to the boundary location information, and use the contextual text as a content element associated with the image; Keywords are extracted from the context text and used as keyword elements associated with the image.

7. The document processing method according to claim 3, characterized in that, When the semantic block is text type, for any semantic block of table, image, or text content type, content elements and keyword elements are extracted according to the semantic extraction strategy matching the content type of the semantic block, including: The continuous text content is divided into multiple paragraphs and filtered to obtain multiple non-empty paragraphs; The content of the multiple non-empty paragraphs is accumulated, and the accumulated paragraph content is used as a content element; Perform word frequency analysis on the accumulated paragraph content, select the most frequent words based on the word frequency analysis results, and use these words as keyword elements; Specifically, if the text length of the current semantic block exceeds the character threshold, the current semantic block is terminated, and the next semantic block is processed.

8. The document processing method according to any one of claims 1 to 7, characterized in that, The semantic elements have a mapping relationship with the field types of the semantic block structure, and the field types include at least a type field, a chapter field, a content field, and a keyword field; the semantic elements are standardized and encapsulated according to the predefined semantic block structure to generate multiple structure instances, including: Create a structure instance for each semantic block, and the structure instance includes field variables such as type field, chapter field, content field and keyword field; Based on the content type of each semantic block, assign values ​​to the type variables in the corresponding structure instance; The chapter elements, content elements, and keyword elements corresponding to each semantic block are assigned to the chapter variables, content variables, and keyword variables in their respective structure instances to complete standardized encapsulation and generate multiple structure instances.

9. The document processing method according to claim 1, characterized in that, The method further includes: Load the semantic vector model, and perform vectorization operations on each structure instance through the semantic vector model to obtain multiple vectorized representations; The multiple vectorized representations are stored sequentially in a vector library for subsequent semantic tasks to call; The order of the multiple vectorized representations is consistent with the order of the semantic blocks corresponding to the respective structure instances in the document to be processed.

10. The document processing method according to claim 9, characterized in that, The semantic task includes at least retrieval tasks and generation tasks; the method further includes: Receive query information containing the user's semantic intent, and perform vectorization processing on the query information to obtain a query vector; Based on the query vector, a search is performed in the vector library to obtain a matching target structure instance; Based on the target structure instance, determine the original content in the document to be processed that matches the query information; wherein the original content is used as the output of the retrieval task and / or as an auxiliary input of the generation task.

11. A document processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the document to be processed; the document to be processed is a portable document format. The parsing module is used to parse the document to be processed page by page to obtain the page content of each page, and to perform semantic block recognition on the content of each page to obtain multiple semantic blocks; The extraction module is used to extract semantic elements from each semantic block according to the modality-aware semantic extraction strategy; The encapsulation module is used to standardize and encapsulate the semantic elements according to a predefined semantic block structure, generating multiple structure instances for association with the original content in the document to be processed in semantic tasks.

12. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the document processing method as described in any one of claims 1-10.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the document processing method as described in any one of claims 1-10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the document processing method as described in any one of claims 1-10.

Images