Intelligent document slicing method and system based on cross-modal attention mechanism

Abstract

The invention provides an intelligent document slicing method and system based on a cross-modal attention mechanism, and the method comprises the steps: obtaining an input document, carrying out the preprocessing of the input document, and extracting the features of a document structure, text semantics, image vision and table structure; taking text features as query, fusing image and table features as key values, calculating a cross-modal association weight through a multi-head attention mechanism, and judging association strength; analyzing a document hierarchy and a topic boundary based on semantic and structural features, and identifying candidate slice points; utilizing a pre-trained reinforcement learning decision model, taking the candidate slice points, the text features, the structural features and the association strength as input, and combining a reward function containing quality, efficiency and consistency dimensions to decide and generate a final slice position sequence; and carrying out length and content optimization on the slices and then outputting a structured result. According to the method, the recognition accuracy of cross-modal image-text and text table association can be remarkably improved, the semantic integrity after slicing is ensured, and the processing efficiency is effectively balanced while the slicing quality is ensured.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to an intelligent document slicing method and system based on a cross-modal attention mechanism. Background Technology

[0002] In the information age, electronic documents, as the core carriers of knowledge and information, have evolved from early plain text formats to complex multimedia composite documents encompassing formatted documents (such as PDF and Word), web page content (HTML), and even integrating text, images, tables, and other elements. This evolution has made the intelligent segmentation of documents with diverse structures and complex content into semantically complete and logically consistent fragments (i.e., "document slices") a crucial foundational issue in fields such as natural language processing, knowledge management, and large-scale model training. The quality of document slices directly determines the upper limit of performance in subsequent tasks such as information retrieval, knowledge extraction, and model pre-training, making its technical importance self-evident.

[0003] To address this challenge, document segmentation technology itself has undergone a profound transformation from simple rule-based methods relying on physical structural features such as paragraph marks and page breaks to logical segmentation based on semantic understanding. While early methods were simple to implement and fast, they heavily relied on the existing regular structure of the document. Subsequently, the development of natural language processing technology spurred the emergence of topic segmentation algorithms, which identify segmentation points by analyzing semantic coherence and topic shifts, demonstrating advantages in processing long documents. In recent years, with the rise of artificial intelligence, especially multimodal learning, multimodal feature fusion methods have emerged, aiming to adapt to the processing needs of modern complex documents by extracting and weightedly fusing features from heterogeneous content such as text, images, and tables.

[0004] However, despite continuous technological iteration, existing methods still face fundamental technical bottlenecks when dealing with real-world documents such as scanned PDFs, technical manuals with mixed text and images, and reports containing complex tables. First, the core issue lies in the weak perception of cross-modal semantic associations. Existing multimodal methods often only superficially integrate information from different modalities, lacking a deep collaborative understanding mechanism. This results in the system's inability to accurately determine the intrinsic connections between images and their explanatory text, or between tables and their interpretive text. Consequently, during segmentation, it is easy to fragment related elements that should be integrated, severely compromising the integrity and readability of the information. Second, the depth and generalization ability of semantic understanding remain insufficient. Whether it's rule-based methods or some shallow semantic models, their accuracy drops significantly when recognizing complex linguistic phenomena such as implicit topic shifts and semantic continuations across paragraphs, leading to inappropriate segmentation boundary selection and the creation of semantically fragmented segments. Furthermore, existing solutions face a dilemma in balancing robustness and efficiency. Rule-based methods are poorly adapted to documents with non-standard structures, while deep learning methods, which have high accuracy potential, are computationally burdensome and cannot meet the stringent requirements of processing speed and resource consumption in practical applications. These shortcomings collectively restrict the further development and application effectiveness of document slicing technology. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an intelligent document slicing method and system based on a cross-modal attention mechanism to solve the problems of weak cross-modal semantic association recognition ability, poor semantic integrity after slicing, and difficulty in balancing processing efficiency and accuracy in the existing technology.

[0006] On one hand, this invention provides an intelligent document slicing method based on a cross-modal attention mechanism, the method comprising: Obtain the input document, perform format recognition and content cleaning on the input document, and obtain standardized document data; The pre-trained encoding model extracts document structure features, text semantic features, image visual features, and table structure features from the normalized document; The text semantic features are used as query vectors, and the image visual features are fused with the table structured features to generate cross-modal reference features as key vectors and value vectors. Through a multi-head attention mechanism, the association weight between the query vector and the key vector is calculated, and the association strength between the text and the image and table is determined based on a preset threshold. Based on the text semantic features and the document structure features, the hierarchical structure of the input document is analyzed and semantic topic boundaries are detected to identify candidate slice points in the input document; Based on the candidate slice points, the text semantic features, the document structure features, and the association strength, a pre-trained reinforcement learning decision model is used to determine the final slice position sequence. The state space of the reinforcement learning decision model includes the positional state of the document processing progress, document structure features, text semantic features, association strength, and boundary information extracted from the candidate slice points. The action space includes executing a slice at the current position, continuing processing without slicing, and backtracking to adjust all slice points. The reward function is constructed based on slice quality, processing efficiency, and multimodal consistency. Based on the final slice position sequence, document slices are obtained, and the slice results are output.

[0007] In some embodiments of the present invention, the input document undergoes format recognition and content cleaning, wherein the format types include portable file formats, document formats, presentation formats, hypertext markup language formats, and plain text file formats; the method further includes: If the input document is identified as being in the portable file format, then optical character recognition is used to extract the text content from the input document. The extracted text content is subjected to normalization operations, which include: merging redundant whitespace characters, correcting heading levels, correcting character recognition errors, and removing headers, footers, and watermark information.

[0008] In some embodiments of the present invention, determining the association strength between text and images or tables based on the preset threshold includes: The association weight is compared with preset strong association thresholds and weak association thresholds; wherein, the association weight is obtained by calculating the cosine similarity or scaled dot product attention score between the query vector and the key vector; If the association weight is greater than the strong association threshold, it is determined to be a strong association; if the association weight is between the weak association threshold and the strong association threshold, it is determined to be a weak association; if the association weight is less than the weak association threshold, it is determined to be no association. Among them, text and image / table elements that are determined to be strongly associated are constrained to remain within the same document slice when determining the final slice position sequence.

[0009] In some embodiments of the present invention, after determining the correlation strength based on the preset threshold, the method further includes: The preset threshold is dynamically adjusted based on the number of pages, structural complexity, and the number of images and tables contained in the input document.

[0010] In some embodiments of the present invention, identifying candidate slice points in the input document includes: Based on the document structure features, the starting position of the title, the section separation position, and the ending position of the paragraph are identified as candidate slice points based on the structure. Based on the semantic features of the text, the semantic boundary positions where the topic changes are detected are used as semantic candidate slice points; The structure-based candidate slice points and the semantic-based candidate slice points are merged to form a set of candidate slice points in the input document.

[0011] In some embodiments of the present invention, the reward function satisfies the following formula: ; in, Represents the reward function; , , These represent the slice quality assessment values. Processing efficiency evaluation value Multimodal consistency evaluation value The weight, and , .

[0012] In some embodiments of the present invention, after obtaining the document slices and before outputting the slice results, the method further includes: The document slices are subjected to content quality checks and repairs; wherein the checks include at least semantic integrity checks, logical coherence checks, and information integrity checks; for slices that fail the checks, repairs are carried out by supplementing the context, adjusting the slice boundaries, or marking them as needing further review.

[0013] In some embodiments of the present invention, after obtaining the document slices and before outputting the slice results, the method further includes: The document slices are length optimized by: merging slices with character lengths less than a minimum threshold with adjacent slices, and / or segmenting slices with character lengths greater than a maximum threshold at semantic boundaries; wherein the minimum threshold and the maximum threshold are positively correlated with the average paragraph length of the input document.

[0014] On the other hand, the present invention also provides an intelligent document slicing method system based on a cross-modal attention mechanism, including a processor, a memory, and a computer program / instructions stored in the memory. The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of any of the methods mentioned above.

[0015] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods mentioned above.

[0016] This invention provides an intelligent document slicing method and system based on a cross-modal attention mechanism. By integrating cross-modal attention with reinforcement learning decision-making, it achieves accurate perception and preservation of the relationships between text and images, and between text and tables, during the document slicing process. This effectively solves the key problems of semantic fragmentation and incorrect segmentation of related elements in existing methods for multimodal content processing. This method can automatically adapt to complex documents with different formats and structures, significantly improving processing efficiency and automation while ensuring the semantic integrity and logical coherence of the slices. It provides high-quality, structured document slicing input for downstream applications such as knowledge management, information retrieval, and large-scale model training, demonstrating strong practicality and wide applicability.

[0017] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0018] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of an intelligent document slicing method based on a cross-modal attention mechanism in one embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram illustrating the process of extracting multi-dimensional features of text, images, and tables using an encoding model in one embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram illustrating the process of calculating the association weights between text, images, and tables based on a multi-head attention mechanism in one embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the process of making intelligent slice decisions based on a reinforcement learning decision model in one embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0024] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0025] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0026] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0027] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0028] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0029] To address the problems of weak cross-modal semantic association recognition capabilities, poor semantic integrity after slicing, and the difficulty in balancing processing efficiency and accuracy in existing technologies, this invention provides an intelligent document slicing method based on a cross-modal attention mechanism, such as... Figure 1 As shown, the method includes the following steps S101~S106: Step S101: Obtain the input document, perform format recognition and content cleaning on the input document, and obtain standardized document data.

[0030] Step S102: Extract document structure features, text semantic features, image visual features, and table structure features from the normalized document using a pre-trained encoding model.

[0031] Step S103: Use the text semantic features as the query vector, and fuse the image visual features with the table structured features to generate cross-modal reference features as key vectors and value vectors; calculate the association weight between the query vector and the key vector through a multi-head attention mechanism, and determine the association strength between the text and the image and table based on a preset threshold.

[0032] Step S104: Based on text semantic features and document structure features, analyze the hierarchical structure of the input document and detect semantic topic boundaries to identify candidate slice points in the input document.

[0033] Step S105: Based on candidate slice points, textual semantic features, document structural features, and association strength, a pre-trained reinforcement learning decision model is used to determine the final slice position sequence. The state space of the reinforcement learning decision model includes the positional state of document processing progress, document structural features, textual semantic features, association strength, and boundary information extracted from candidate slice points. The action space includes options to slice at the current position, continue processing without slicing, and backtrack to adjust all slice points. The reward function is constructed based on slice quality, processing efficiency, and multimodal consistency.

[0034] Step S106: Obtain document slices based on the final slice position sequence and output the slice results.

[0035] like Figure 2 The diagram shows the steps of an intelligent document slicing method based on a cross-modal attention mechanism.

[0036] The core objective of step S101 is to transform heterogeneous, complex, and potentially noisy original input documents into unified, standardized, and high-quality plain text and structural information data, providing a reliable foundation for subsequent feature extraction and intelligent decision-making.

[0037] In step S101, the core task is to perform format recognition and content cleaning and standardization on the input document.

[0038] In some embodiments, the system automatically identifies the file format of the input document and performs targeted processing for each file format, specifically including: For PDF documents, a portable file format is typically used, automatically recognizing and distinguishing between text PDFs and scanned PDFs. For scanned PDFs, an optical character recognition (OCR) engine is automatically invoked to extract the text while preserving the layout information.

[0039] For Word documents, typically in .doc / .docx format, the document object model is parsed to extract the text stream, while preserving original formatting metadata such as fonts, paragraphs, and headings.

[0040] For PowerPoint documents, i.e., presentation format, extract the text content and title structure of each slide.

[0041] For HTML web pages, the Hypertext Markup Language (HTML) format is typically used to parse the DOM tree, remove non-text scripts and tags such as navigation bars and advertisements, and extract the core text content.

[0042] For plain text files, the encoding can be read directly.

[0043] In some embodiments, after basic text extraction is completed, the document content is input into a pre-trained document classification model to automatically identify the business or domain type to which it belongs.

[0044] The document classification model can employ deep neural networks, such as the Transformer classifier, trained on massive labeled document datasets. It can determine the document domain based on word choice, sentence structure, and structural features. Specifically, the input data is the text content of the document, such as its title, abstract, the first N paragraphs, or the preprocessed full text. The output is the corresponding business type label for the document, such as technical document, business document, academic document, and training material. Technical documents refer to API documents, technical specifications, development guidelines, etc.; business documents refer to requirement documents, product manuals, user manuals, etc.; academic documents refer to academic papers, research reports, etc.; and training materials refer to tutorials, operation guides, etc.

[0045] In some embodiments, a series of automated cleaning and repair operations are performed on the extracted raw text stream, specifically including: Merge consecutive whitespace characters (such as spaces and tabs), standardize newline characters, and remove meaningless spaces at the beginning and end of lines.

[0046] Based on formatting information such as font size, bolding, and numbering, the system automatically detects heading levels. For missing or incorrect levels (such as jumping directly from "Heading 1" to "Heading 3"), it completes or corrects them according to preset rules or machine learning models to ensure that the heading hierarchy is continuous and logically correct.

[0047] For common character errors generated during OCR or transmission (such as recognizing the number "0" as the letter "O" or "rn" as "m"), a pre-trained language model combined with a domain-specific dictionary is used to calculate the most likely correct character and replace it. The domain-specific dictionary can be selected based on the business type of the document.

[0048] Automatically identify and remove noise information unrelated to the main semantics of the document, such as headers, footers, page numbers, watermarks, and footnote numbers.

[0049] The semantic vector similarity of adjacent or similar paragraphs is calculated. When the similarity exceeds the threshold, it is determined to be duplicate content. The system will delete the redundant parts and retain the version with the most complete semantics or the earliest appearance.

[0050] For sentences that have been incorrectly cut due to line breaks or column splits, the system analyzes the syntactic structure and semantic coherence, and automatically merges them without changing the original meaning, thus restoring the complete sentence.

[0051] For text fragments whose confidence level is still below the preset threshold after the above automatic correction (such as text in a blurry image), the system does not force modification, but inserts a specific placeholder mark and adds it to the list of texts to be manually reviewed, so as to ensure the reliability of automated processing.

[0052] The core objective of step S102 is to transform the heterogeneous content such as text, images, and tables in the standardized document data obtained in the previous step into a unified numerical feature representation that can be deeply understood by machines, thereby providing an information foundation for subsequent association modeling and decision-making.

[0053] In step S102, as Figure 2 As shown, document structure features, text semantic features, image visual features, and table structure features are extracted from normalized documents through multiple pre-trained encoding models.

[0054] In some embodiments, plain text content from a normalized document is input into a pre-trained language model. This language model uses BERT, ERNIE, RoBERTa, or variants thereof as its backbone network and generates semantic vectors of different granularities through hierarchical encoding, including: The context-related vector representation of each word or subword (usually corresponding to the hidden state of the last layer of the pre-trained language model) is used as a word-level feature.

[0055] By using pooling operations, such as average pooling and [CLS] tag vectors, the overall semantic vector of a sentence or paragraph can be obtained, thus obtaining sentence / paragraph level features.

[0056] By aggregating the full-text vectors or using a dedicated document-level coding layer, we obtain vectors that represent the overall theme and style of the document, i.e., document-level features.

[0057] In some embodiments, structural information such as heading hierarchy (e.g., nesting relationships of H1, H2, H3), paragraph organization (paragraph order and grouping), and list structure (ordered / unordered lists and their bullets) corrected in the preprocessing stage is obtained. This structural information is encoded into feature vectors. For example, the heading hierarchy (1,2,3...) of the current text unit (e.g., sentence) is converted into an embedding vector, binary features are used to represent "whether it is in the list" and "whether it is the beginning / end of a paragraph", a tree structure representation of paragraphs or chapters is constructed, and graph features are extracted. These feature representation methods are used to parse and quantify the layout and logical structure information from the standardized document data to obtain document structural features.

[0058] In some embodiments, visual elements such as images and charts embedded in the document are subjected to feature extraction using a pre-trained computer vision model, specifically including: Locate all image regions in the document and crop them into individual image blocks.

[0059] Each image patch is fed into a pre-trained convolutional neural network (such as ResNet, Vision Transformer) or a cross-modal model (such as CLIP's image encoder).

[0060] Extract the global average pooling vector from the high-level feature map of the convolutional neural network, or the [CLS] label vector of ViT, as the visual semantic feature vector of the image.

[0061] In some embodiments, structural parsing and content understanding are performed on tables in a document to extract their features, specifically including: Use specialized table recognition models (such as TableNet, CascadeTabNet) or rule-based methods to determine table boundaries and identify the physical layout of rows, columns, and cells, especially handling merged cells.

[0062] First, the text within each cell is extracted. Then, the table's row and column structure, header information, and data type (number, text) are encoded as features. For example, a feature vector is generated for each cell, containing its row and column index, whether it is a header cell, and merge information. Finally, the linearized text of the table (such as the cell content read row by row) is input into a lightweight language model to obtain the overall semantic summary vector of the table. The table's structured features include this semantic summary vector and encoded features such as row and column structure.

[0063] In some embodiments, if the confidence level of the table recognition is too low, the original image chunks are retained and marked as requiring further special processing or manual review.

[0064] Based on step S102, the normalized document data is deconstructed and encoded into a set of aligned, numerical features: document structural features, text semantic features, image visual features, and table structural features.

[0065] The core objective of step S103 is to accurately model and quantify the semantic association strength between text and images, and between text and tables in a document, and to quantify this association into categories with clear decision-making guidance significance, thereby solving the fundamental problem of incorrect segmentation of multimodal elements in the prior art.

[0066] In step S103, a computational framework centered on text and actively querying and associating visual and tabular information is constructed based on the multi-head attention mechanism of the Transformer architecture. The specific implementation can be divided into two stages: association weight calculation and association strength determination.

[0067] First, an attention calculation module is constructed, whose inputs are a query vector (text semantic features), a key vector, and a value vector (cross-modal reference features). Then, a standard multi-head attention operation is used. For each text segment (query vector), the system calculates the scaled dot product attention score between it and all fused cross-modal reference features (key vectors). After attention calculation, one or more association weight scores are output for each text unit (e.g., paragraph).

[0068] In some embodiments, image visual features are fused with table structured features to generate cross-modal reference features as key and value vectors. This can be achieved through unified fusion or dual-path parallel processing. Specifically: Unified fusion path: Input the image visual feature vector set and the table structured feature vector set (which may be linearized or pooled) into a lightweight fusion network (such as several fully connected layers), and output a unified cross-modal reference feature sequence representing all non-textual information, which serves as the key vector and value vector in the attention mechanism, respectively.

[0069] Dual-path parallel operation: Two independent attention heads or sub-modules run in parallel, namely the image-text attention module and the text-table attention module. In the image-text attention module, text semantic features are used as query vectors, and image visual features are used as key and value vectors. In the text-table attention module, text semantic features are used as query vectors, and table structured features are used as key and value vectors.

[0070] Under the unified fusion path, a comprehensive weight is output; under the dual-parallel path, "text-image weight" and "text-table weight" are output respectively. This weight is a scalar value; the higher the value, the stronger the semantic association between the text and its corresponding image / table.

[0071] In some embodiments, the calculated continuous association weight scores are discretized into association categories with clear downstream operational implications based on a preset threshold range, to obtain association strength, such as strong association, weak association, and no association. Specific steps include: Preset strong correlation thresholds and weak correlation thresholds, with the strong correlation threshold being greater than the weak correlation threshold.

[0072] For each text-image weight and text-table weight (collectively referred to as association weight), determine the degree of association: If the association weight is greater than the strong association threshold, it is determined to be a strong association, indicating that the image / table is the core explanation, direct example, or inseparable data display of the text. Text and image / table elements determined to be strongly associated are constrained to remain within the same document slice when determining the final slice position sequence.

[0073] If the association weight is between the weak association threshold and the strong association threshold, it is determined to be a weak association, indicating that the image / table is related to the text topic, but may not be a direct correspondence, or it may be background information.

[0074] If the association weight is less than the weak association threshold, it is determined to be unrelated.

[0075] In some embodiments, the strong correlation threshold and weak correlation threshold are dynamically adjusted based on the number of pages, structural complexity, and number of images and tables contained in the input document.

[0076] The core objective of step S104 is to plan all reasonable and potential action options, i.e. candidate slice points, for the subsequent reinforcement learning agent based on the understood content (semantics and structure), so that the agent can learn and make decisions efficiently and accurately.

[0077] In some embodiments, based on document structural features, the explicit layout and organizational tags of the document are directly parsed and utilized to locate natural physical or logical dividing points, specifically including: Identify the starting character position of all headings (such as H1, H2, H3). Headings often mark the beginning of a new chapter or topic and are strong candidate slicing points.

[0078] Identify explicit chapter separators in a document, such as page breaks, section breaks, etc., or specific chapter markers in certain document formats.

[0079] Identify the end point of each paragraph. A paragraph is the smallest unit that expresses a complete meaning, and its end is a basic candidate point.

[0080] For ordered or unordered lists, identify the end position of each list item.

[0081] These boundaries are based on the document's syntax or layout, are explicit, objective, and easily detectable, and can provide a first-level, high-confidence set of candidate locations for slicing.

[0082] In some embodiments, the semantic evolution of text content is analyzed in depth based on text semantic features to detect the locations where the topic naturally shifts, specifically including: The document is represented as a sequence of semantic feature vectors of sentences or short paragraphs, and the cosine similarity or semantic distance between adjacent semantic units is calculated. When the similarity drops significantly (a "valley" appears), it indicates that there has been a jump in topic between the preceding and following content, and this position is marked as a potential semantic boundary.

[0083] A pre-trained topic segmentation model, such as one based on BERT, is acquired and fine-tuned for document segmentation tasks to predict sentence or paragraph-level boundary probabilities. This topic segmentation model takes the entire document's textual semantic feature sequence as input and outputs a series of topic boundary probabilities. The system selects the peak positions where the probability exceeds a preset threshold as the semantic topic boundaries.

[0084] Track the appearance and disappearance of core entities (such as names and technical terms) and topic keywords in documents. When the dominant entity / topic keyword set changes significantly, it can be inferred that a topic shift has occurred.

[0085] These boundaries, based on the semantics of the document, identify turning points in the internal logic of the content. They can pinpoint locations where there are no obvious formatting marks, but the content has undergone substantial changes; for example, there may be no heading between two paragraphs, but the first paragraph provides background information while the second begins to discuss the implementation plan.

[0086] By merging, deduplicating, and sorting the two types of boundary points, a unified set of candidate slice points is obtained.

[0087] Step S105 is the central decision engine of the intelligent document slicing method of this invention, simulating the process of an intelligent agent reading and segmenting a document: the agent reads the document from left to right, and at each decision moment, based on its current understanding (state) of the document, selects one of several optional actions to execute, with the goal of obtaining the highest cumulative reward when completing the processing of the entire document. This reward represents the comprehensive score of the final slicing result in terms of quality, efficiency, and consistency.

[0088] The implementation of step S105 revolves around a pre-trained reinforcement learning decision model. The training and use of this model follow the Markov decision process framework, which includes three main elements: state, action, and reward.

[0089] First, we will explain the modeling of the environment and interaction process.

[0090] The decision model processes documents iteratively. Starting from the beginning of the document, at each decision step, the model observes the current state (i.e., the feature vector defined in the state space), selects an action from the action space and executes it. Based on the action execution result (e.g., if a slice is executed, a slice boundary is generated; if the option to continue is selected, the processing position is moved forward), the environment (i.e., the document processing system) updates the internal processing state and calculates the immediate reward.

[0091] Then, the three elements of the decision-making model will be explained.

[0092] The state space is a summary of all environmental information perceived by the model at each decision step. A state is a multi-dimensional feature vector, including: Location feature: The ratio of the current processing position to the total document length.

[0093] Document structure features: current heading level, paragraph type (body text / list item), list status, etc.

[0094] Textual semantic features: topic distribution of the current location and its context, conceptual coherence, semantic similarity with the preceding and following text, etc.

[0095] Association strength: Whether there is a strong or weak association between the current location and the image / table.

[0096] Boundary information extracted from candidate slice points, such as how far away from the next candidate point.

[0097] The action space is the set of discrete operations that the decision model can perform in each state, specifically including: Execute slice at current position (slice_here): Start a new slice with the current position (which must be one of the candidate slice points) as the end boundary of a slice.

[0098] Continue processing without slicing: Instead of slicing, move the processing position forward to the next decision point (the beginning of the next sentence or paragraph) and continue the analysis.

[0099] Adjusting previous slices: This is an optimization action. When the model deems the previous slice selection unsatisfactory, it can backtrack and modify the position of the previous slice boundary, for example, by moving it to a better candidate point.

[0100] The reward function is a quantitative metric used to evaluate the quality of a single action or the final slice sequence. In this invention, the reward function comprehensively considers three dimensions: slice quality, processing efficiency, and consistency.

[0101] The quality dimension is used to evaluate the semantic integrity, structural consistency, and informational completeness of the generated slices. For example, does a slice represent a complete subtopic and does it contain all necessary reference elements?

[0102] The efficiency dimension is used to evaluate processing speed and whether the slice length is within a preset reasonable range. Slices that are too short or too long will be penalized.

[0103] The consistency dimension focuses on the integrity of multimodal elements. It heavily rewards decisions that keep strongly related text / image or text / table elements within the same slice, while severely penalizing decisions that split them. The integrity of citation relationships is also considered.

[0104] In some embodiments, the reward function satisfies the following formula (1): ; (1) in, Represents the reward function; , , These represent the slice quality assessment values. Processing efficiency evaluation value Multimodal consistency evaluation value The weight, and , .

[0105] In some embodiments, the model is trained using a dataset containing a large number of documents and their manually annotated or automatically generated high-quality slice locations. The training process specifically includes: The decision model employs a deep Q-network or policy network and is trained using a large number of document samples.

[0106] An experience playback mechanism is adopted to randomly sample historical decision-making experiences (state, action, reward, next state) to form batches for learning.

[0107] The target network is updated regularly to stabilize the training process.

[0108] An ε-greedy strategy is adopted, which explores actions randomly with a high probability in the early stage of training, and gradually increases the probability of selecting the highest-scoring action in the later stage.

[0109] In some embodiments, the application method of the trained decision model specifically includes: For a new input document, the model selects actions step by step from the initial state based on the above interaction process (usually using a greedy strategy, that is, always selecting the action that the model considers to be of the highest value) until the entire document is processed.

[0110] All the positions where the slice (slice_here) action is executed at the current position are sequentially arranged to form the final slice position sequence.

[0111] Based on step S105, an optimal or near-optimal slice position sequence is finally obtained.

[0112] The core task of step S106 is to slice and optimize the document based on the final slice position sequence generated by the decision model, and organize it into a structured data format for output, so as to ensure that the slices delivered to downstream applications (such as knowledge bases, large model training, retrieval systems) have the quality of being directly usable.

[0113] In some embodiments, after document slicing, the segmented fragments are optimized, including both length optimization and content quality optimization.

[0114] Length optimization ensures that the length of each document slice is within a preset, reasonable range, balancing readability and information density. Specific steps include: Iterate through all the initial slices obtained from the final slice position sequence and calculate the number of characters or words in each slice.

[0115] For slices shorter than a minimum threshold (e.g., 200 characters), they are merged with their semantically most relevant adjacent slices (either the preceding or following one). Semantic relevance is checked before merging; for example, the cosine similarity of the text vectors of the two slices is calculated. Only slices with high relevance are merged to avoid forcibly merging irrelevant content.

[0116] For slices exceeding the maximum threshold (e.g., 2000 characters), semantic boundaries are searched within them for further segmentation. Utilizing candidate slice point information generated in previous steps, or by re-analyzing the semantic structure within the long slice (e.g., paragraphs, subheadings), it is split into multiple shorter sub-slices without compromising semantic integrity.

[0117] Slices within a reasonable length are retained directly.

[0118] The content quality optimization algorithm performs in-depth quality diagnosis and repair on each slice to ensure its semantic consistency, logical coherence, and information completeness. Specifically, this includes: Each slice is checked in three dimensions: semantic integrity, logical coherence, and information integrity.

[0119] Among them, semantic integrity check is used to analyze whether the slice expresses a relatively complete subtopic or concept, and whether the start and end are within reasonable semantic unit (such as paragraph) boundaries.

[0120] The logical coherence check is used to check whether the sentences within a slice are logically coherent and whether there are any breaks in reference caused by forced segmentation.

[0121] The information integrity check is used to verify whether all the image and text elements and table elements marked as strongly related in the previous steps are indeed fully included in the slice, and to check whether other key data and conclusions have been omitted.

[0122] In some embodiments, one or more of the following measures are taken for slices that fail the inspection: Automatically extract a small number of key contextual sentences (such as the preceding text) from adjacent slices and append them to the current slice.

[0123] Fine-tune the start and end positions of the slice, for example, by expanding / shrinking it forward or backward by several sentences to include complete semantic units.

[0124] For issues that cannot be automatically repaired (such as complex logical breaks), the system marks them as requiring manual review in the slice metadata and adds them to the quality control checklist.

[0125] Finally, the optimized document slices and their rich metadata are packaged and output in a machine-readable and easy-to-use format.

[0126] Corresponding to the above method, the present invention also provides an intelligent document slicing method system based on a cross-modal attention mechanism. The system includes a processor, a memory, and a computer program / instructions stored in the memory. The processor executes the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the aforementioned method. This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0127] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0128] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0129] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart document slicing method based on a cross-modal attention mechanism, characterized in that, The method includes: Obtain the input document, perform format recognition and content cleaning on the input document, and obtain standardized document data; The pre-trained encoding model extracts document structure features, text semantic features, image visual features, and table structure features from the normalized document; The text semantic features are used as query vectors, and the image visual features are fused with the table structured features to generate cross-modal reference features as key vectors and value vectors. Through a multi-head attention mechanism, the association weight between the query vector and the key vector is calculated, and the association strength between the text and the image and table is determined based on a preset threshold. Based on the text semantic features and the document structure features, the hierarchical structure of the input document is analyzed and semantic topic boundaries are detected to identify candidate slice points in the input document; Based on the candidate slice points, the text semantic features, the document structure features, and the association strength, a pre-trained reinforcement learning decision model is used to determine the final slice position sequence. The state space of the reinforcement learning decision model includes the positional state of the document processing progress, document structure features, text semantic features, association strength, and boundary information extracted from the candidate slice points. The action space includes executing a slice at the current position, continuing processing without slicing, and backtracking to adjust all slice points. The reward function is constructed based on slice quality, processing efficiency, and multimodal consistency. Based on the final slice position sequence, document slices are obtained, and the slice results are output.

2. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, The input document undergoes format recognition and content cleaning, wherein the format types include portable file formats, document formats, presentation formats, hypertext markup language formats, and plain text file formats; the method further includes: If the input document is identified as being in the portable file format, then optical character recognition is used to extract the text content from the input document. The extracted text content is subjected to normalization operations, which include: merging redundant whitespace characters, correcting heading levels, correcting character recognition errors, and removing headers, footers, and watermark information.

3. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, Determining the correlation strength between text and images / tables based on the preset threshold includes: The association weight is compared with preset strong association thresholds and weak association thresholds; wherein, the association weight is obtained by calculating the cosine similarity or scaled dot product attention score between the query vector and the key vector; If the association weight is greater than the strong association threshold, it is determined to be a strong association; if the association weight is between the weak association threshold and the strong association threshold, it is determined to be a weak association; if the association weight is less than the weak association threshold, it is determined to be no association. Among them, text and image / table elements that are determined to be strongly associated are constrained to remain within the same document slice when determining the final slice position sequence.

4. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, After determining the correlation strength based on the preset threshold, the method further includes: The preset threshold is dynamically adjusted based on the number of pages, structural complexity, and the number of images and tables contained in the input document.

5. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, Identifying candidate slice points in the input document includes: Based on the document structure features, the starting position of the title, the section separation position, and the ending position of the paragraph are identified as candidate slice points based on the structure. Based on the semantic features of the text, the semantic boundary positions where the topic changes are detected are used as semantic candidate slice points; The structure-based candidate slice points and the semantic-based candidate slice points are merged to form a set of candidate slice points in the input document.

6. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, The reward function satisfies the following formula: ； in, Represents the reward function; , , These represent the slice quality assessment values. Processing efficiency evaluation value Multimodal consistency evaluation value The weight, and , .

7. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, After obtaining the document slices but before outputting the slice results, the process also includes: The document slices are subjected to content quality checks and repairs; wherein the checks include at least semantic integrity checks, logical coherence checks, and information integrity checks; for slices that fail the checks, repairs are carried out by supplementing the context, adjusting the slice boundaries, or marking them as needing further review.

8. The intelligent document slicing method based on cross-modal attention mechanism according to claim 1, characterized in that, After obtaining the document slices but before outputting the slice results, the process also includes: The document slices are length optimized by: merging slices with character lengths less than a minimum threshold with adjacent slices, and / or segmenting slices with character lengths greater than a maximum threshold at semantic boundaries; wherein the minimum threshold and the maximum threshold are positively correlated with the average paragraph length of the input document.

9. A system for intelligent document slicing based on a cross-modal attention mechanism, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

Images