Document processing method and device, equipment and medium

Abstract

The invention discloses a document processing method and device, equipment and a medium, which are used for quickly and accurately processing a document to the greatest extent. According to the method, for a to-be-processed document, layout information of the document can be identified firstly; based on the layout information, preliminarily partitioning the document to obtain preliminarily partitioned text blocks; then, aiming at every two adjacent initially-divided text blocks, determining a model on the basis of association relationship characteristics between the two initially-divided text blocks and a pre-trained text connection relationship, and determining whether to merge the two initially-divided text blocks to obtain each bipartite text block contained in the document; according to the method, vector coding is performed on each bipartite text block to obtain the comprehensive vector of the document, so that the purpose of quickly and accurately processing the document to the greatest extent can be achieved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a document processing method, apparatus, device and medium. Background Technology

[0002] With the widespread adoption of digital office tools, enterprises and organizations have accumulated a large amount of documents, including PDF documents, Word documents, Excel spreadsheets, PowerPoint presentations, and other formats. How to process these documents efficiently to extract and utilize information has become a significant challenge for knowledge management and intelligent question-answering systems.

[0003] In related technologies, when processing document data (hereinafter referred to as document for ease of description), the document is usually segmented (blocked) based on a fixed length or simple delimiters. This coarse segmentation method is prone to destroying the semantic integrity of the document, thereby restricting the practicality and accuracy of knowledge management and intelligent question answering systems.

[0004] Therefore, there is an urgent need for a technical solution that can process documents as quickly and accurately as possible. Summary of the Invention

[0005] This application provides a document processing method, apparatus, device, and medium for processing documents with maximum speed and accuracy.

[0006] Firstly, this application provides a document processing method, the method comprising: For the document to be processed, the layout information of the document is identified; based on the layout information, the document is initially divided into blocks to obtain initial text blocks; For each pair of adjacent initial text segments, the correlation features between the two initial text segments are determined. The correlation features are input into a pre-trained text connection relationship determination model. Based on the output of the text connection relationship determination model, it is determined whether to merge the two initial text segments; thus, the bipartite text segments contained in the document are obtained. Each of the two-part text blocks is vector-encoded to obtain the comprehensive vector of the document.

[0007] In one possible implementation, determining whether to merge the two adjacent text blocks based on the output of the text connection relationship determination model includes: Based on the text connection relationships, the model output contains connection probabilities and a pre-configured connection probability threshold suitable for the document, and determines whether to merge the two adjacent text blocks.

[0008] In one possible implementation, the configuration process for the connection probability threshold includes: Identify the target document type of the document; Based on the pre-saved correspondence between document types and content coherence threshold adjustment values, the target threshold adjustment value corresponding to the target document type is determined; Based on the pre-saved base threshold and the target threshold adjustment value, a suitable connection probability threshold for the document is determined.

[0009] In one possible implementation, the initial segmentation of the document includes: Based on optical character recognition (OCR), each coarse text block bounding box of the document is obtained; and based on the layout detection model, the layout bounding box of each layout component of the document is obtained. For each bold text block, determine the layout box to which the bold text block belongs based on its positional relationship with each layout box; The document is initially divided into blocks based on each layout box.

[0010] In one possible implementation, determining the association features between the two initially segmented text blocks includes: Based on at least one of the positional relationship features, layout relationship features, semantic relationship features, and lexical relationship features between the two initially segmented text blocks, the association relationship features between the two initially segmented text blocks are determined.

[0011] In one possible implementation, the vector encoding of each of the binary text blocks includes: For each bipartite text block, based on OCR, the first text length and the first vector of the bipartite text block are obtained; and based on the visual large model, the second text length and the second vector of the bipartite text block are obtained; based on the first text length and the second text length, the first weight corresponding to the first vector and the second weight corresponding to the second vector are determined respectively; based on the weighted sum of the first vector, the first weight, the second vector, and the second weight, the vector of the bipartite text block is obtained.

[0012] In one possible implementation, the composite vector includes the location information vectors of each of the two-part text blocks.

[0013] In one possible implementation, after obtaining each of the bipartite text blocks contained in the document and before performing vector encoding on each of the bipartite text blocks, the method further includes: For each binary text block, determine whether to split the binary text block according to the pre-configured text block length threshold; if the binary text block needs to be split, then split the binary text block according to the pre-configured delimiter priority and the delimiters contained in the binary text block, and then perform the subsequent vector encoding steps on the split binary text block.

[0014] Secondly, this application provides a document processing apparatus, the apparatus comprising: The initial segmentation module is used to identify the layout information of the document to be processed; based on the layout information, the document is initially segmented into blocks to obtain initial text blocks; The binary segmentation module is used to determine the correlation features between two adjacent initial text segments, input the correlation features into a pre-trained text connection relationship determination model, and determine whether to merge the two initial text segments based on the output of the text connection relationship determination model; thus obtaining each binary text segment contained in the document. The vectorization module is used to perform vector encoding on each of the binary text blocks to obtain the comprehensive vector of the document.

[0015] In one possible implementation, the binary module is specifically used for: Based on the text connection relationships, the model output contains connection probabilities and a pre-configured connection probability threshold suitable for the document, and determines whether to merge the two adjacent text blocks.

[0016] In one possible implementation, the binary module is specifically used for: Identify the target document type of the document; Based on the pre-saved correspondence between document types and content coherence threshold adjustment values, the target threshold adjustment value corresponding to the target document type is determined; Based on the pre-saved base threshold and the target threshold adjustment value, a suitable connection probability threshold for the document is determined.

[0017] In one possible implementation, the initial division module is specifically used for: Based on optical character recognition (OCR), each coarse text block bounding box of the document is obtained; and based on the layout detection model, the layout bounding box of each layout component of the document is obtained. For each bold text block, determine the layout box to which the bold text block belongs based on its positional relationship with each layout box; The document is initially divided into blocks based on each layout box.

[0018] In one possible implementation, the binary module is specifically used for: Based on at least one of the positional relationship features, layout relationship features, semantic relationship features, and lexical relationship features between the two initially segmented text blocks, the association relationship features between the two initially segmented text blocks are determined.

[0019] In one possible implementation, the vectorization module is specifically used for: For each bipartite text block, based on OCR, the first text length and the first vector of the bipartite text block are obtained; and based on the visual large model, the second text length and the second vector of the bipartite text block are obtained; based on the first text length and the second text length, the first weight corresponding to the first vector and the second weight corresponding to the second vector are determined respectively; based on the weighted sum of the first vector, the first weight, the second vector, and the second weight, the vector of the bipartite text block is obtained.

[0020] In one possible implementation, the composite vector includes the location information vectors of each of the two-part text blocks.

[0021] In one possible implementation, the vectorization module is specifically used for: For each binary text block, determine whether to split the binary text block according to the pre-configured text block length threshold; if the binary text block needs to be split, then split the binary text block according to the pre-configured delimiter priority and the delimiters contained in the binary text block, and then perform the subsequent vector encoding steps on the split binary text block.

[0022] Thirdly, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory stores a computer program that, when executed by the processor, causes the processor to perform the method described in any of the first aspects. Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0023] Fifthly, this application provides a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform the steps of the method described in any of the first aspects.

[0024] This application addresses the document to be processed by first identifying its layout information. Based on this layout information, the document is initially segmented into blocks, resulting in initial text blocks. Then, for each pair of adjacent initial text blocks, a model is determined based on the correlation features between the two blocks and pre-trained text connection relationships to decide whether to merge them, thus obtaining the document's individual binary text blocks. By vector encoding each binary text block, a comprehensive vector for the document is obtained. Because intelligent segmentation combines document layout information and semantic awareness, the completeness, accuracy, and speed of document information extraction are maximized, thereby achieving the goal of processing documents quickly and accurately to the greatest extent possible. Attached Figure Description

[0025] To more clearly illustrate the implementation methods in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0026] Figure 1 A schematic diagram of a document processing process provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating another document processing procedure provided in an embodiment of this application; Figure 3 A schematic diagram of a document processing device provided in an embodiment of this application; Figure 4 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0028] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0029] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0030] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0031] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0032] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0033] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0034] Furthermore, the data collection, dissemination, and use in the technical solution of this application all comply with the requirements of relevant national laws and regulations.

[0035] To process documents as quickly and accurately as possible, this application provides a document processing method, apparatus, device, and medium. Preferred embodiments of this application will be described below with reference to the accompanying drawings.

[0036] Example 1: See Figure 1 , Figure 1 This application provides a schematic diagram of a document processing process, which includes the following steps: S101: For the document to be processed, identify the layout information of the document; based on the layout information, perform preliminary segmentation of the document to obtain initial text blocks.

[0037] The document processing method provided in this application is applied to electronic devices, such as personal computers (PCs), mobile terminals, servers, etc., and this application does not specifically limit them.

[0038] In one possible implementation, when processing a document, its layout information can be identified, i.e., layout recognition can be performed. This layout recognition can utilize computer vision technology to identify the semantic types of different layout components in the document, such as titles, body text, tables, images, headers, and footers, providing a foundation for structured document parsing. Specifically, based on the layout recognition results, the document can be initially segmented into blocks. For example, each layout component can be used as a basic segmentation unit to obtain the text blocks contained in the document (referred to as initial text blocks for ease of description).

[0039] In one possible implementation, when initially segmenting the document, optical character recognition (OCR) can be used to obtain each coarsely segmented text block (also known as a text region) contained in the document. OCR is a technology that converts text in an image into machine-editable text; it can recognize the character content and location information of a document image using a deep learning model.

[0040] OCR recognition can be performed in two steps: detecting text regions and recognizing the characters contained within the text regions. For text region detection, deep learning-based object detection algorithms can be used. For example, a convolutional neural network optimized with ONNXRuntime (ORT) can be used for text region detection. This can output the coordinates of the bounding box (hereinafter referred to as the coarse text block box) for each text region, as well as the confidence score corresponding to the bounding box coordinates of each text region.

[0041] When recognizing characters contained in a text region, image preprocessing processes such as denoising and size normalization can be performed on each detected text region to convert pixel coordinates into relative coordinates. A pre-trained model based on the Transformer model architecture is used for character recognition, outputting the character sequence and its corresponding coordinate information. Post-processing operations such as removing noisy characters, correcting erroneous text, and standardizing text format are also performed. The output of the entire process includes the recognized text (character) content and the precise coordinate position of each character or word in the image, which will not be elaborated here.

[0042] Simultaneously, based on the layout detection model (a convolutional neural network object detection framework), it can identify (detect) the bounding boxes (referred to as layout boxes) of different types of layout components in a document, such as body text, titles, images, tables, annotations, headers and footers, references, and formulas. During the layout box identification process, non-maximum suppression (NMS) can be used to remove overlapping detection boxes, ensuring the uniqueness of the region containing each layout component. Each identified layout box region is labeled with its corresponding semantic type, such as body text or title, and its corresponding bounding box coordinates are generated, which will not be elaborated further here.

[0043] After obtaining each coarse text block and layout box, a spatial association matching process can be performed. Specifically, a spatial association algorithm can be used to match the text content recognized by OCR with the layout detection results obtained by the layout detection model, establishing a correspondence between region types (layout boxes) and text content. Specifically, for each coarse text block, the layout box to which the coarse text block belongs can be determined based on its positional relationship with each layout box, thus establishing a correspondence between the text content within the coarse text block and its associated layout box. Based on each layout box, the document can be initially segmented into blocks.

[0044] The specific spatial association matching strategy is as follows: For each coarse text block frame identified by OCR, traverse all layout frames (also known as layout regions) contained in the document, and calculate the intersection-union ratio (CUI) (the ratio of the intersection area to the union area) between the coarse text block frame and each layout frame. A higher CUI indicates a higher degree of overlap, and the layout frame with the highest CUI is identified as the layout frame to which the coarse text block frame belongs, thus the text content in the coarse text block is considered to belong to that layout frame. Alternatively, for each coarse text block frame, after calculating the CUI with each layout frame, determine whether the CUI is greater than a set CUI threshold (e.g., 0.5). If one or more CUIs are greater than the threshold, the layout frame with the highest CUI (i.e., the layout frame with the highest overlap) is identified as the layout frame to which the coarse text block frame belongs. If there is no intersection-union ratio greater than the threshold, the layout box that is closest to the text block can be determined as the layout box to which the text block belongs, based on the positional relationship between the text block and each layout box.

[0045] S102: For each pair of adjacent initial text blocks, determine the correlation features between the two initial text blocks, input the correlation features into a pre-trained text connection relationship determination model, and determine whether to merge the two initial text blocks based on the output of the text connection relationship determination model; thus obtaining each of the bipartite text blocks contained in the document.

[0046] In one possible implementation, considering that documents in formats such as PDF or Word, after the initial segmentation described above, result in discrete text blocks with unclear logical relationships, while adjacent layout components (text blocks) may have semantic connections—for example, adjacent images and their captions, tables and table titles, etc.—can be merged into a single complete text block. This improves semantic integrity and ensures complete contextual information is available during subsequent retrievals. Therefore, to improve the accuracy of document processing, intelligent segmentation logic can be used again to determine whether two adjacent initial text blocks can be merged, thus achieving intelligent segmentation (secondary segmentation) of the initial text blocks.

[0047] Specifically, during secondary segmentation (merging initial text blocks), for each pair of adjacent initial text blocks, the correlation features between the two initial text blocks can be determined first. Then, these correlation features can be input into a pre-trained text connection relationship determination model. Based on the output of the text connection relationship determination model, it is determined whether to merge the two initial text blocks. In this way, by traversing all the initial text blocks, the secondary segmentation of the document is completed, and the text blocks contained in the document after secondary segmentation (for ease of description, they are called binary text blocks) are obtained.

[0048] In determining the relationship features between two adjacent initial text blocks, one or more of the following can be used: positional relationship features, layout relationship features, semantic relationship features, lexical relationship features, etc.

[0049] Specifically, the positional relationship features between two adjacent initial text blocks can include one or more of the following: vertical distance features, horizontal position features, page span features, column affiliation features, paragraph first-line indentation features, and line spacing pattern features.

[0050] The vertical distance feature between two adjacent initial text blocks can be used to characterize whether these two text blocks belong to a continuous paragraph. If they belong to a continuous paragraph, it is more likely that the two initial text blocks will be merged. To determine the vertical distance feature between two adjacent initial text blocks, the ratio between the vertical spacing (vertical distance) of the two text blocks and the maximum height of the two text blocks (for ease of description, this is called the vertical distance ratio) can be calculated first. Then, the vertical distance ratio can be compared with a preset vertical distance ratio threshold. If the vertical distance ratio is less than the preset threshold, it can be considered that the two text blocks are likely to belong to a continuous paragraph; otherwise, it can be considered that the two text blocks are likely not to belong to a continuous paragraph. Here, we assume that text block 1 and text block 2 are two adjacent text blocks. In the vertical direction, each text block contains a top edge (also called the top edge) and a bottom edge (also called the bottom edge). The height of a text block is the difference between the vertical coordinates of its bottom edge and its top edge. Assuming that text block 2 is below text block 1, the vertical distance ratio between the two text blocks can be calculated based on the following formula: Vertical distance ratio = (vertical coordinates of the top edge of text block 2 - vertical coordinates of the bottom edge of text block 1) / max(height of text block 1, height of text block 2).

[0051] The horizontal positional features between two adjacent initial text blocks can be used to characterize the horizontal alignment relationship between the two text blocks, including whether the two text blocks are left-aligned, right-aligned, or center-aligned. Compared to the difference in the horizontal alignment of two text blocks, when the horizontal alignment of two text blocks is the same, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged.

[0052] The page span feature between two adjacent initial text blocks can be used to characterize whether the two text blocks are located on the same page. Compared to two text blocks being located on different pages, when two text blocks are located on the same page, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged.

[0053] The column affiliation feature between two adjacent initial text blocks can be used to characterize the column affiliation relationship between the two text blocks in a multi-column layout. Compared to two text blocks belonging to different columns, when two text blocks belong to the same column, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged.

[0054] The paragraph first-line indentation feature between two adjacent initial text blocks can be used to characterize the consistency and regularity of the paragraph first-line indentation of the two text blocks. Compared with the different paragraph first-line indentation methods of two text blocks, when the paragraph first-line indentation methods of two text blocks are the same, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge these two text blocks (initial text blocks).

[0055] The line spacing pattern between two adjacent initial text blocks can be used to characterize whether the two text blocks conform to the pattern of belonging to the same paragraph. Compared with the line spacing patterns of two adjacent text blocks being different, when the line spacing patterns of two text blocks are the same, it can be considered that these two text blocks are more likely to belong to the semantically continuous text blocks of the same paragraph, and it is more inclined to merge these two text blocks (initial text blocks).

[0056] Specifically, the layout relationship features (also known as layout type features) between two adjacent initial text blocks can characterize the hierarchical relationship between the two text blocks in terms of chapters, headings, and numbering. For example, the hierarchical relationship between two text blocks in terms of chapters, headings, and numbering can be identified through various predefined regular expression patterns. The layout relationship features between two adjacent initial text blocks can include one or more of the following: layout consistency features, specific layout identifier features, layout transition pattern features, directory structure distance features, paragraph nesting depth features, image and text mixing relationship features, hyperlink density features, and text density distribution features.

[0057] The layout consistency feature between two adjacent initial text blocks can be used to characterize whether the two text blocks belong to the same chapter or the same layout component type (e.g., whether they are both body text or both are headings). Compared to two adjacent text blocks not belonging to the same chapter or having different layout types (layout inconsistency), when two adjacent text blocks belong to the same chapter and have the same layout type (layout consistency), it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged.

[0058] The specific layout identifier features between two adjacent initial text blocks can be used to characterize the consistency of Boolean features for specific layout types such as text and tables. Compared to the inconsistency of Boolean features for specific layout types between two adjacent text blocks, when the Boolean features for specific layout types of two adjacent text blocks are consistent, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged.

[0059] The layout transition pattern feature between two adjacent initial text blocks can be used to characterize whether the two text blocks follow a layout transition pattern such as from title to body text, or from body text to table. Compared to two adjacent text blocks having inconsistent layout transition patterns, when two adjacent text blocks have consistent layout transition patterns, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged.

[0060] The directory structure distance feature between two adjacent initially segmented text blocks can be used to characterize the path distance between the two text blocks in the document directory tree. When the path distance between two adjacent text blocks in the document directory tree is less than a set path distance threshold, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge these two text blocks (initial segmented text blocks). Conversely, when the path distance between two adjacent text blocks in the document directory tree is not less than the set path distance threshold, it can be considered that these two text blocks are more likely not to be semantically continuous text blocks, and it is more inclined not to merge these two text blocks (initial segmented text blocks).

[0061] The paragraph nesting depth feature between two adjacent initial text blocks can be used to characterize the consistency of the nesting level of the two text blocks in the document structure. Compared with the different nesting levels of two adjacent text blocks in the document structure, when two adjacent text blocks have the same nesting level in the document structure, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge these two text blocks (initial text blocks), and vice versa, which will not be elaborated further.

[0062] The image-text mixed layout feature between two adjacent initial text blocks can be used to characterize whether the two text blocks correspond to an image and its explanatory text (mixed layout relationship). For example, when one of the two adjacent text blocks is an image and the other is text, it can be identified whether the distance between the two text blocks is less than a set distance threshold. If it is less, it can be considered that the two text blocks are likely to correspond to an image and its corresponding explanatory text, and it is more likely that the two text blocks (initial text blocks) will be merged. The reverse is also true, and so on.

[0063] The hyperlink density feature between two adjacent initial text blocks can be used to characterize whether the hyperlink densities contained in the two text blocks are the same or similar. Compared to a large difference in the hyperlink densities contained in two text blocks, when the hyperlink densities contained in two text blocks are the same or similar (the difference is less than a set difference threshold), it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge these two text blocks (initial text blocks).

[0064] The text density distribution features between two adjacent initially segmented text blocks can be used to characterize whether the text density distributions contained in the two text blocks are the same or similar. Compared to text density distributions that differ significantly between two text blocks, when the text density distributions contained in two text blocks are the same or similar (the difference is less than a set difference threshold), it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge these two text blocks (initial segmented text blocks).

[0065] Specifically, the semantic relationship features (also known as semantic connection features) between two adjacent initially segmented text blocks can include one or more of the following: sentence integrity features, grammatical connectivity features, bracket matching features, punctuation pattern features, semantic role continuity features, and keyword overlap features.

[0066] Among them, the sentence integrity feature between two adjacent initial text blocks can be used to characterize whether the two adjacent text blocks are both complete sentences. For example, regular expressions can be used to identify whether two adjacent text blocks both end with a period, question mark, exclamation mark, or other closing character. If so, it can be considered that the sentence integrity of the two adjacent text blocks is relatively good, and it can be considered that these two text blocks are more likely not to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will not be merged; the opposite is also true.

[0067] The syntactic connectivity features between two adjacent initial text blocks can be used to characterize whether the latter text block in two adjacent text blocks begins with a conjunction, preposition, or other word indicating continuation. If so, it can be considered that the two text blocks are more likely to be semantically continuous text blocks, and it is more likely that the two text blocks (initial text blocks) will be merged; the opposite is also true.

[0068] The bracket matching feature between two adjacent initial text blocks can be used to characterize whether each of the two adjacent text blocks contains a half bracket that can be paired with brackets. For example, if the first text block contains a left bracket and the second text block contains a right bracket, then these two text blocks are more likely to be semantically continuous text blocks, and it is more likely that these two text blocks (initial text blocks) will be merged; and vice versa.

[0069] The punctuation pattern features between two adjacent initial text blocks can be used to characterize whether the first text block ends with a punctuation mark such as a comma or semicolon, or whether the second text block begins with a punctuation mark such as a comma or semicolon. If so, it can be considered that the two text blocks are more likely to be semantically continuous text blocks, and it is more likely that the two text blocks (initial text blocks) will be merged; otherwise, it is not.

[0070] The semantic role continuity feature between two adjacent initial text blocks can be used to characterize whether semantic roles such as subject, predicate, and object exist across two text blocks. For example, whether the previous text block only contains the subject and the next text block only contains the predicate and object, etc., so that these semantic roles are complete across text blocks. If so, it can be considered that these two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge these two text blocks (initial text blocks); the opposite is also true.

[0071] Keyword overlap between two adjacent initial text blocks can be used to characterize the similarity of keywords contained in the two text blocks. For example, keywords contained in the two text blocks can be extracted separately, and the Jaccard similarity coefficient of the keywords contained in the two text blocks can be calculated. When the Jaccard similarity coefficient is greater than a set similarity coefficient threshold, it can be considered that the keywords of the two text blocks are relatively similar, and it can be considered that the two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge the two text blocks (initial text blocks); and vice versa.

[0072] Specifically, the lexical relationship features between two adjacent initially segmented text blocks may include one or more of the following: token overlap features, part-of-speech features, proper noun continuity features, merged word increment features, and word quantity difference features.

[0073] The token overlap feature between two adjacent initial text blocks can be used to characterize the degree of overlap at the word level at the block boundary. Tokens can refer to basic units in the text, such as words and punctuation marks. When the token overlap between two text blocks exceeds a set overlap threshold, these two text blocks are considered more likely to be semantically continuous, and the initial text blocks are more likely to be merged; conversely, the lower the overlap, the lower the overlap. The token overlap can be obtained by calculating the intersection-union ratio (IUGR) between the number of words in the last sentence of the first text block (hereinafter referred to as the last words of text block 1) and the number of words in the first sentence of the second text block (hereinafter referred to as the first words of text block 2). This IUGR is expressed by the formula: Token overlap = |last words of text block 1 ∩ first words of text block 2| / |last words of text block 1 ∪ first words of text block 2|.

[0074] The part-of-speech features between two adjacent initially segmented text blocks can be used to characterize the part-of-speech distribution features of different types of words, such as nouns, verbs, and adjectives, at the boundary between the two text blocks. For example, natural language processing techniques can be used to analyze the lexical distribution features of two text blocks. When the lexical distribution features of the last sentence at the end of the first text block are the same as or similar to the lexical distribution features of the first sentence at the beginning of the second text block, it can be considered that the two text blocks are more likely to be semantically continuous, and it is more likely that the two text blocks (initial segmented text blocks) will be merged; the reverse is also true.

[0075] The continuity feature of proper nouns between two adjacent initial text blocks can be used to characterize the distribution of proper nouns such as personal names, place names, and organization names in the two text blocks. When the distribution of proper nouns in the two text blocks is the same or similar, it can be considered that the two text blocks are more likely to be semantically continuous, and it is more likely that the two text blocks (initial text blocks) will be merged; the opposite is also true.

[0076] The lexical increment feature after merging two adjacent initially segmented text blocks can be used to characterize the difference between the number of words in the text formed after merging and deduplicating the two text blocks and the original total number of words in the two text blocks. When the absolute value of this difference is greater than a set difference threshold, it can be considered that the reduction in the number of words after merging the two text blocks is relatively large, and it can be considered that the two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge the two text blocks (initially segmented text blocks); and vice versa.

[0077] The word count difference feature between two adjacent initially segmented text blocks can be used to characterize the difference in word count between the two text blocks. Optionally, when the word count difference between the two text blocks is particularly large or particularly small (greater than the first difference threshold or less than the second difference threshold), it can be considered that the two text blocks are more likely to be semantically continuous text blocks, and it is more inclined to merge the two text blocks (initial segmented text blocks). When the word count difference (difference) between the two text blocks is neither particularly large nor particularly small (between the first and second difference thresholds), it can be considered that the two text blocks are more likely not to be semantically continuous text blocks, and it is more inclined not to merge the two text blocks (initial segmented text blocks).

[0078] In one possible implementation, after obtaining the aforementioned association features, these features can be input into a pre-trained text connection determination model. The model analyzes these features and outputs the probability that the two text blocks can be connected (connection probability). Based on the model's output connection probability and a pre-configured connection probability threshold suitable for the document, it can be determined whether to merge the two adjacent text blocks. In one possible implementation, if the model's output connection probability is greater than the pre-configured connection probability threshold, the two text blocks are considered to be merged; conversely, if the model's output connection probability is not greater than the pre-configured connection probability threshold, the two text blocks are considered not to be merged.

[0079] In one possible implementation, the text connectivity determination model can be a Gradient Boosting Decision Tree (GBDT) model. GBDT is an ensemble learning algorithm that constructs a strong predictive model by combining multiple weak decision tree learners, achieving high-precision prediction through the ensemble of these weak learners. For example, its output connectivity probability P(connection|feature) can be expressed as: P(connection|feature) = sigmoid( × (Features)), among which, Let be the weight of the i-th decision tree. Let N be the output of the i-th decision tree, and N be the total number of decision trees.

[0080] In one possible implementation, the gradient boosting tree model (text connection determination model) can be trained in advance using a large number of labeled sample documents. This allows the model to learn the probability distribution of merging (connecting) two text blocks under different feature combinations. The model can employ a binary classification design, outputting a connection probability score between 0 and 1. The training process of the text connection determination model is described below.

[0081] When training the text connection relationship determination model, the sample set contains sample association relationship features between two text blocks of multiple sample documents. Each sample association relationship feature may contain one or more of the following: positional relationship features, layout relationship features, semantic relationship features, and lexical relationship features between the two initially segmented text blocks. In addition, each sample association relationship feature corresponds to a label indicating whether it can be merged (for ease of description, it is called a connection probability label).

[0082] When training the text connection relationship determination model, the association relationship feature of any sample in the sample set can be obtained. The obtained sample association relationship feature is input into the text connection relationship determination model to be trained. Through the text connection relationship determination model to be trained, the connection probability corresponding to the sample association relationship feature is obtained (for ease of description, it is called the recognition connection probability).

[0083] In practice, the accuracy of the text connection determination model's recognition result can be determined by whether the connection probability labels match the recognized connection probabilities. If they do not match, it indicates that the recognition result of the text connection determination model is inaccurate, and the parameters of the text connection determination model need to be adjusted to train the model.

[0084] In practice, when adjusting the parameters in the text connection relationship determination model, the gradient descent algorithm can be used to backpropagate the gradients of the parameters of the text connection relationship determination model, thereby training the text connection relationship determination model.

[0085] In one possible implementation, the above operation can be performed on each sample association feature in the sample set. When the preset convergence condition is met, the text connection relationship is determined and the model training is completed.

[0086] The preset convergence conditions can include: the sample association features in the sample set are used to determine the model through the text connection relationship to be trained; the number of correctly identified sample association features is greater than a set number; or the number of iterations for training the text connection relationship determination model reaches the set maximum number of iterations. These conditions can be flexibly set in practice and are not specifically limited here.

[0087] In one possible implementation, in order to improve the generalization ability of the model, the training data sample set can cover a variety of document types such as academic papers, technical documents, news reports, and legal documents, and this application does not make any specific limitations on this.

[0088] In one possible implementation, considering that different document types and application scenarios have different requirements for determining whether text blocks can be merged and connected, in order to improve the flexibility and accuracy of document processing, the connection probability threshold can be configured to be dynamically adjusted according to different document types. For example, when parsing a document, the user can choose whether the document type is continuous or independent. For example, for documents with strong logic, such as academic papers, the independence between paragraphs is stronger, so they can be set to independent, and the appropriate connection probability threshold can be configured higher to retain more independent text blocks and avoid erroneously merging unrelated paragraphs. On the other hand, for literary works such as novels and essays, the content is highly coherent and the paragraphs are closely connected. Too many blocks may destroy its semantic integrity, so they can be set to continuous, and the appropriate connection probability threshold can be configured lower to make the text blocks easier to merge and connect, avoiding the incorrect segmentation of continuous plots and affecting its semantic integrity. Specifically, when determining the appropriate connection probability threshold for the current document to be processed, the target document type can be identified first. Then, based on the pre-saved correspondence between document types and content coherence threshold adjustment values, the target threshold adjustment value corresponding to the target document type can be determined. Based on the pre-saved base threshold and target threshold adjustment value, the appropriate connection probability threshold for the document can be determined.

[0089] The formula for adjusting the appropriate connection probability threshold for a document is: Connection Probability Threshold = Base Threshold + Content Coherence Threshold Adjustment Value (Target Threshold Adjustment Value). For example, the base threshold can be 0.5, the threshold adjustment value for independent documents can be 0.1, and their appropriate connection probability threshold can be 0.6. The threshold adjustment value for continuous documents can be -0.1, and their appropriate connection probability threshold can be 0.4.

[0090] Compared to using a fixed threshold, this application can adapt to the characteristics of different types of documents and dynamically adjust the connection probability threshold, thereby avoiding over-segmentation or over-merging to the greatest extent and improving the accuracy of document processing.

[0091] If the connection probability of two initially segmented text blocks is greater than the document's connection probability threshold, these two initially segmented text blocks can be merged; otherwise, they can remain separate. In this way, the various binary text blocks contained in the document can be obtained.

[0092] S103: Perform vector encoding on each of the two-part text blocks to obtain the comprehensive vector of the document.

[0093] In one possible implementation, to improve document processing speed, after obtaining the bipartite text blocks contained in the document and before performing vector encoding on each bipartite text block, it can be determined whether to split the bipartite text block based on a pre-configured text block length threshold. For example, it can be determined whether the number of tokens contained in the bipartite text block is greater than a pre-configured text block length threshold (token number threshold, such as 512-2048 tokens). If it is greater, the bipartite text block can be considered too long. To improve document processing efficiency, the bipartite text block can be split, and the subsequent step of vector encoding of the bipartite text block can be performed based on the split bipartite text blocks.

[0094] When splitting a binary text block, the split can be based on pre-configured delimiter priorities and the delimiters contained within the block. For example, it can prioritize splitting at strong delimiters (periods, question marks, exclamation marks), followed by weak delimiters (semicolons, colons), and finally at commas and other symbols. This approach maximizes the semantic integrity of each text block, minimizes splitting in the middle of sentences or at key information points, and ensures a moderate text block size for efficient document processing. The text block length threshold (token quantity threshold) can be flexibly set according to requirements; this application does not impose specific limitations on it.

[0095] Vectorization refers to the process of converting unstructured data such as text and images into high-dimensional numerical vectors, enabling computers (electronic devices) to perform mathematical operations and similarity calculations. To facilitate the use of documents by knowledge management and intelligent question-answering systems, each binary text block in the document can be vectorized to obtain a comprehensive document vector. Knowledge management and intelligent question-answering operations can then be performed based on this comprehensive document vector. The process of performing knowledge management and intelligent question-answering operations based on the document vectors can utilize existing technologies and will not be elaborated upon here. The process of obtaining the comprehensive document vector is described below.

[0096] In one possible implementation, when obtaining the comprehensive vector of a document, i.e., when performing document vectorization, different encoding strategies can be adopted for different document types, such as whether the document is query text or document text. This is the process of using differentiated encoding strategies for text vectorization. For example, for query text, unique query instructions can be used to encode each binary text block contained in the document. For instance, an instruction-based encoding such as "generate a representation for this sentence to retrieve related articles" can be used to encode each binary text block, thereby enhancing the retrieval guidance of the generated query vector. Then, the sub-vectors of each binary text block can be concatenated to obtain the comprehensive vector of the document. For document text, each binary text block contained in the document can be directly encoded to maintain the original semantic representation of the document content.

[0097] In one possible implementation, to improve the accuracy and flexibility of the obtained document composite vector, multimodal content can be enabled or disabled during document vectorization. When multimodal content is enabled for document vectorization, for the image content in each bipartite text block in the document, OCR technology can be used to extract the text information s1 from the image of that bipartite text block, obtaining the first text length of that bipartite text block (denoted as len1 for ease of description). The first vector of that bipartite text block (denoted as v1 for ease of description) is then obtained by encoding and vectorizing the text information in the image of that bipartite text block. Simultaneously, visual features of the image of that bipartite text block can be obtained using a large visual model, yielding textual description information s2 of the visual features. This allows the second text length of that bipartite text block (denoted as len2 for ease of description) to be obtained. The second vector of that bipartite text block (denoted as v2 for ease of description) is then obtained by encoding and vectorizing the textual description information of the visual features of the image of that bipartite text block. The vector (also called sub-vector) of the bipartite text block can be obtained by weighted summation of the first and second vectors; then, the comprehensive vector of the document can be obtained by combining and merging the vectors of each bipartite text block.

[0098] In determining the weights of the first and second vectors, to improve flexibility and accuracy, the first weight of the first vector and the second weight of the second vector can be dynamically adjusted based on the richness of the text content in the image. For example, the first weight of the first vector can be expressed as len1 / (len1+len2), and the second weight of the second vector can be expressed as len2 / (len1+len2). Then, the vectors of the bipartite text blocks in the document can be obtained based on the weighted sum of the first vector, first weight, second vector, and second weight, thus obtaining the comprehensive vector of the document. Based on this, when the text content is richer, the text vector (first vector) can have a larger weight, and the comprehensive vector can rely more on the text vector (first vector). Conversely, when the text content is less rich, the visual feature vector (second vector) can have a larger weight, and the comprehensive vector can rely more on the visual feature vector. This maximizes the ability to combine the weights of both text vectors and visual feature vectors for matching when performing vector retrieval based on the comprehensive vector (such as knowledge management and intelligent question answering), thereby maximizing the practicality and accuracy of the knowledge management and intelligent question answering system.

[0099] Compared to traditional plain text extraction methods, this application can combine the document's layout structure and multimodal content to improve the completeness of information extraction and the accuracy of vector representation, especially for technical documents containing complex tables and mixed text and graphics, where the effect is more significant.

[0100] In one possible implementation, to improve the accuracy and efficiency of vector retrieval, the overall vector of the document can include (add) position information vectors for each binary text block. Specifically, position information encoding (position information vector) can be added to each binary text block, including structured information such as the page number of the binary text block in the document, its relative position on the page, and the chapter it belongs to. For each binary text block, its position information vector can be concatenated and fused with its content vector (such as the first and second vectors described above) to obtain the vector (sub-vector) of that binary text block. Then, by concatenating and fusing the vectors of each binary text block, the overall vector of the document can be obtained.

[0101] By adding location information vectors to the composite vector, the structural hierarchy of the document can be utilized during subsequent vector retrieval. The location-aware vector representation can help improve the accuracy of retrieval, especially when dealing with long documents, it can better locate relevant content.

[0102] To facilitate understanding, the document processing procedure provided in this application will be explained below through a specific embodiment. (See reference...) Figure 2 , Figure 2This application provides another document processing procedure diagram, which includes the following steps: S201: For the document to be processed, identify the document's format, parse the document based on the parser corresponding to the document's format, identify the document's layout information, and perform preliminary segmentation of the document based on the layout information to obtain initial text blocks.

[0103] Regular expressions can be used to match and identify document extensions, thereby recognizing the document's format. Document formats can include: PDF, DOCX (Word), PowerPoint (PPT), Excel, Markdown, JSON, TXT, etc. This application does not impose specific limitations on the document format.

[0104] When parsing PDF documents using a PDF parser, the PDF document can first be converted into an image. A scaling factor can be used to convert each page of the document into a high-resolution image, ensuring the accuracy of subsequent OCR recognition. Deep learning models can be used for OCR text recognition and location, followed by the identification of layout structure information (including text, titles, tables, images, etc.). If the document contains tables, a specialized table structure recognition algorithm can be activated to identify the table areas and extract their images. A deep learning-based table structure analysis model is then used to identify the table's row and column structure, cell boundaries, and spanning relationships. Afterward, the table content is reconstructed. This table content reconstruction process converts the two-dimensional table structure into a linearized natural language description, including row and column relationships, cell content, and header information. Then, a specially designed table encoding template is used to convert the structured information into a natural language description, for example, following a description pattern of "table title - row title - column title - cell content". This structured text description preserves the logical relationships of the table while facilitating understanding and encoding by vectorized models. Based on this, it is possible to parse PDF documents with complex layouts, accurately identify table structures, and precisely parse multimodal content such as text, images, and tables in PDF documents. It can also identify the layout information (layout structure information) of PDF documents and perform preliminary segmentation of the document based on the layout information to obtain initial text blocks.

[0105] When parsing docx format documents based on a DOCX parser, it can extract text content and paragraph formatting information by traversing paragraphs and table elements in the document, identify embedded images and tables in the document, organize content according to the document layout structure, maintain the original layout structure of the document, and support intelligent table parsing and image and text association processing.

[0106] When parsing PPT documents using a PPT parser, the slides can be traversed in page order to extract various shapes and text content from text boxes, tables and grouped objects can be processed, and the content can be organized in reading order. Each slide's content is treated as a text block. When a text block exceeds the maximum token limit, it can be split according to pre-configured delimiter priorities and the delimiters contained within it; this will not be elaborated further here.

[0107] When parsing XLSX and CSV documents using an Excel parser, the workbook can be loaded, all worksheets in the document can be traversed, headers and data rows can be extracted, and the documents can be converted into structured text or HTML format. The documents can be divided into blocks according to the number of rows or the maximum number of tokens threshold to obtain segmented text blocks. The complete association between headers and data rows can be maintained, and large tables can be processed in blocks.

[0108] When parsing Markdown formatted documents using a Markdown parser, the document can be parsed based on predefined regular expressions, Markdown tables and headings can be identified and extracted, tables can be separated from other content, the document can be divided into blocks according to the maximum token quantity threshold, and the Markdown format can be preserved.

[0109] When parsing JSON documents using a JSON parser, a recursive, structured processing approach can be used to parse the JSON structure, convert the list into a dictionary format, divide the document into chunks based on a maximum token count threshold, and maintain the original nesting relationships.

[0110] When parsing txt format documents using a TXT parser, the text encoding can be detected, and the document can be divided into blocks (segments) based on the user-configured delimiter priority and the delimiters contained in the document. The size of the text blocks can be controlled by limiting the maximum number of tokens.

[0111] In one possible implementation, considering that the initial text blocks obtained from the layout information of documents in formats such as PPT, Excel, Markdown, JSON, and TXT are relatively independent and have relatively weak semantic connectivity, vector encoding can be directly performed on these initial text blocks to obtain the comprehensive vector of the document. The initial text blocks of documents in formats such as PDF and DOCX have relatively strong semantic connectivity, and these formats can proceed to subsequent steps S202-S203.

[0112] This application can be based on parsers such as PDF parser, DOCX parser, PPT parser, EXCEL parser, JSON parser, Markdown parser, and TXT parser, supporting the parsing of various file formats such as PDF, DOCX, EXCEL, PPT, Markdown, and TXT. It can achieve accurate parsing of complex format documents through deep learning models, and use different parsing strategies for the characteristics of different document formats, thereby improving document parsing accuracy, significantly improving the accuracy of document layout recognition, and improving the speed and accuracy of document processing.

[0113] Compared to related technologies that typically only support a single format or use a general parsing method, this application can fully utilize the structural characteristics of different document formats, significantly improving document parsing accuracy. At the same time, it does not require modification of the core logic of document processing when adding new document types, demonstrating good scalability.

[0114] S202: For each pair of adjacent initial text blocks, determine the correlation features between the two initial text blocks, input the correlation features into a pre-trained text connection relationship determination model, and determine whether to merge the two adjacent text blocks based on the connection probability contained in the output of the text connection relationship determination model and the pre-configured connection probability threshold suitable for the document; obtain each of the bipartite text blocks contained in the document.

[0115] Among them, the association relationship features between the two initially segmented text blocks can be determined based on one or more of the positional relationship features, layout relationship features, semantic relationship features, and lexical relationship features.

[0116] The configuration process for the connection probability threshold can be as follows: identify the target document type of the document; determine the target threshold adjustment value corresponding to the target document type based on the pre-saved correspondence between document types and content coherence threshold adjustment values; and determine the connection probability threshold suitable for the document based on the pre-saved base threshold and target threshold adjustment value.

[0117] S203: For each bipartite text block, based on OCR, obtain the first text length and first vector of the bipartite text block; and based on the visual large model, obtain the second text length and second vector of the bipartite text block; based on the first text length and second text length, determine the first weight corresponding to the first vector and the second weight corresponding to the second vector, respectively; based on the weighted sum of the first vector, the first weight, the second vector, and the second weight, obtain the vector of the bipartite text block. Simultaneously, the positional information vectors of each bipartite text block can be concatenated and fused into the vector of the bipartite text block. By concatenating and fusioning the vectors of each bipartite text block, a comprehensive vector of the document is obtained.

[0118] In one possible implementation, to improve vectorization efficiency, the document vectorization process can be based on an intelligent batch processing mechanism. For example, text blocks can be grouped according to a preset batch size (the maximum number of text blocks that can be processed in parallel simultaneously). Simultaneously, longer text blocks can be truncated according to a pre-configured text block length threshold during vectorization. For instance, text blocks can be truncated based on pre-configured delimiter priorities and the delimiters contained within them. This maximizes document processing efficiency while preserving the semantic integrity of the text blocks, meeting the needs of large-scale document processing.

[0119] Additionally, the batch size (the maximum number of text blocks that can be processed in parallel simultaneously) can be automatically adjusted and optimized based on the GPU memory usage and the distribution of text block lengths. For example, when there is ample remaining GPU memory and the text block lengths are all small, the batch size (the maximum number of text blocks that can be processed in parallel simultaneously) can be configured to be larger; conversely, when there is limited remaining GPU memory and the text block lengths are all large, the batch size (the maximum number of text blocks that can be processed in parallel simultaneously) can be configured to be smaller. This application does not impose specific limitations on this.

[0120] The document processing method provided in this application supports unified vectorized representation of multimodal content such as text, images, and tables. It fully utilizes the structural information of documents and is applicable to various practical application scenarios, such as enterprise knowledge management, legal document processing, and education and training. In enterprise knowledge management, large enterprise internal document libraries typically contain a large number of multimodal documents, including technical specifications, operation manuals, and training materials. When processing these documents using the method described in this application, the method can accurately understand charts, flowcharts, and technical diagrams, providing employees with precise knowledge retrieval and question-and-answer services, significantly improving work efficiency. In legal document processing, legal documents contain numerous tables, seals, signatures, and other key visual elements. The method described in this application, through deep document understanding technology, can accurately identify and understand the legal meaning of these elements, providing lawyers and legal professionals with professional document analysis and retrieval support. In education and training, teaching materials contain rich multimodal teaching elements such as charts, formulas, and experimental images. The method described in this application can accurately understand this multimodal content, providing students and teachers with intelligent learning guidance and teaching resource retrieval services.

[0121] This application addresses the document to be processed by first identifying its layout information. Based on this layout information, the document is initially segmented into blocks, resulting in initial text blocks. Then, for each pair of adjacent initial text blocks, a model is determined based on the correlation features between the two blocks and pre-trained text connection relationships to decide whether to merge them, thus obtaining the document's individual binary text blocks. By vector encoding each binary text block, a comprehensive vector for the document is obtained. Because intelligent segmentation combines document layout information and semantic awareness, the completeness, accuracy, and speed of document information extraction are maximized, thereby achieving the goal of processing documents quickly and accurately to the greatest extent possible.

[0122] Example 2: Based on the same technical concept, this application provides a document processing apparatus, see reference. Figure 3 , Figure 3 This application provides a schematic diagram of a document processing apparatus, which includes: The initial segmentation module 301 is used to identify the layout information of the document to be processed; and based on the layout information, to perform initial segmentation of the document to obtain initial text blocks. The binary segmentation module 302 is used to determine the correlation features between two adjacent initial text segments, input the correlation features into a pre-trained text connection relationship determination model, and determine whether to merge the two initial text segments based on the output of the text connection relationship determination model; thus obtaining each binary text segment contained in the document. The vectorization module 303 is used to perform vector encoding on each of the binary text blocks to obtain the comprehensive vector of the document.

[0123] In one possible implementation, the binary module 302 is specifically used for: Based on the text connection relationships, the model output contains connection probabilities and a pre-configured connection probability threshold suitable for the document, and determines whether to merge the two adjacent text blocks.

[0124] In one possible implementation, the binary module 302 is specifically used for: Identify the target document type of the document; Based on the pre-saved correspondence between document types and content coherence threshold adjustment values, the target threshold adjustment value corresponding to the target document type is determined; Based on the pre-saved base threshold and the target threshold adjustment value, a suitable connection probability threshold for the document is determined.

[0125] In one possible implementation, the initial division module 301 is specifically used for: Based on optical character recognition (OCR), each coarse text block bounding box of the document is obtained; and based on the layout detection model, the layout bounding box of each layout component of the document is obtained. For each bold text block, determine the layout box to which the bold text block belongs based on its positional relationship with each layout box; The document is initially divided into blocks based on each layout box.

[0126] In one possible implementation, the binary module 302 is specifically used for: Based on at least one of the positional relationship features, layout relationship features, semantic relationship features, and lexical relationship features between the two initially segmented text blocks, the association relationship features between the two initially segmented text blocks are determined.

[0127] In one possible implementation, the vectorization module 303 is specifically used for: For each bipartite text block, based on OCR, the first text length and the first vector of the bipartite text block are obtained; and based on the visual large model, the second text length and the second vector of the bipartite text block are obtained; based on the first text length and the second text length, the first weight corresponding to the first vector and the second weight corresponding to the second vector are determined respectively; based on the weighted sum of the first vector, the first weight, the second vector, and the second weight, the vector of the bipartite text block is obtained.

[0128] In one possible implementation, the composite vector includes the location information vectors of each of the two-part text blocks.

[0129] In one possible implementation, the vectorization module 303 is specifically used for: For each binary text block, determine whether to split the binary text block according to the pre-configured text block length threshold; if the binary text block needs to be split, then split the binary text block according to the pre-configured delimiter priority and the delimiters contained in the binary text block, and then perform the subsequent vector encoding steps on the split binary text block.

[0130] Based on the same technical concept, this application also provides an electronic device. Figure 4 This application provides a schematic diagram of an electronic device structure, such as... Figure 4 As shown, it includes: processor 401, communication interface 402, memory 403 and communication bus 404, wherein processor 401, communication interface 402 and memory 403 communicate with each other through communication bus 404. The memory 403 stores a computer program, which, when executed by the processor 401, causes the processor 401 to perform the steps of any of the above methods.

[0131] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0132] Communication interface 402 is used for communication between the above-mentioned electronic device and other devices.

[0133] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0134] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0135] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, it causes the electronic device to execute the steps of the above-described method.

[0136] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0137] Based on the same technical concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the above embodiments. Since the principle by which the above computer program product solves the problem is similar to that of the above method embodiments, the implementation of the above computer program product can refer to the implementation of the method, and repeated details will not be described again.

[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A document processing method, characterized in that, The method includes: For the document to be processed, the layout information of the document is identified; based on the layout information, the document is initially divided into blocks to obtain initial text blocks; For each pair of adjacent initial text segments, the correlation features between the two initial text segments are determined. The correlation features are input into a pre-trained text connection relationship determination model. Based on the output of the text connection relationship determination model, it is determined whether to merge the two initial text segments; thus, the bipartite text segments contained in the document are obtained. Each of the two-part text blocks is vector-encoded to obtain the comprehensive vector of the document.

2. The method according to claim 1, characterized in that, The step of determining whether to merge two adjacent text blocks based on the output of the model determined by the text connection relationship includes: Based on the text connection relationships, the model output contains connection probabilities and a pre-configured connection probability threshold suitable for the document, and determines whether to merge the two adjacent text blocks.

3. The method according to claim 2, characterized in that, The configuration process for the connection probability threshold includes: Identify the target document type of the document; Based on the pre-saved correspondence between document types and content coherence threshold adjustment values, the target threshold adjustment value corresponding to the target document type is determined; Based on the pre-saved base threshold and the target threshold adjustment value, a suitable connection probability threshold for the document is determined.

4. The method according to claim 1, characterized in that, The initial segmentation of the document includes: Based on optical character recognition (OCR), each coarse text block bounding box of the document is obtained; and based on the layout detection model, the layout bounding box of each layout component of the document is obtained. For each bold text block, determine the layout box to which the bold text block belongs based on its positional relationship with each layout box; The document is initially divided into blocks based on each layout box.

5. The method according to claim 1, characterized in that, The determination of the association features between the two initially segmented text blocks includes: Based on at least one of the positional relationship features, layout relationship features, semantic relationship features, and lexical relationship features between the two initially segmented text blocks, the association relationship features between the two initially segmented text blocks are determined.

6. The method according to claim 1, characterized in that, The vector encoding of each of the two-part text blocks includes: For each bipartite text block, based on OCR, the first text length and the first vector of the bipartite text block are obtained; and based on the visual large model, the second text length and the second vector of the bipartite text block are obtained; based on the first text length and the second text length, the first weight corresponding to the first vector and the second weight corresponding to the second vector are determined respectively; based on the weighted sum of the first vector, the first weight, the second vector, and the second weight, the vector of the bipartite text block is obtained.

7. The method according to any one of claims 1-6, characterized in that, The composite vector contains the position information vectors of each of the two-part text blocks.

8. The method according to any one of claims 1-6, characterized in that, After obtaining each bipartite text block contained in the document, and before performing vector encoding on each bipartite text block, the method further includes: For each binary text block, determine whether to split the binary text block according to the pre-configured text block length threshold; if the binary text block needs to be split, then split the binary text block according to the pre-configured delimiter priority and the delimiters contained in the binary text block, and then perform the subsequent vector encoding steps on the split binary text block.

9. A document processing device, characterized in that, The device includes: The initial segmentation module is used to identify the layout information of the document to be processed; based on the layout information, the document is initially segmented into blocks to obtain initial text blocks; The binary segmentation module is used to determine the correlation features between two adjacent initial text segments, input the correlation features into a pre-trained text connection relationship determination model, and determine whether to merge the two initial text segments based on the output of the text connection relationship determination model; thus obtaining each binary text segment contained in the document. The vectorization module is used to perform vector encoding on each of the binary text blocks to obtain the comprehensive vector of the document.

10. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program that, when executed by the processor, causes the processor to perform the method according to any one of claims 1-8.

Images