A multi-modal adaptation device for low-resource multilingual document parsing

Abstract

The application provides a kind of multi-modal adaptation device for low-resource multilingual document analysis, comprising: visual encoder, by progressive block merging, image is hierarchically encoded, a set of feature maps is generated, and projection and spatial alignment are carried out, to generate the visual feature map of uniform space grid, while extracting page-level description vector through global average pooling;Adaptive visual-text mapping module, the visual feature map is decomposed into horizontal, vertical, diagonal and global four kinds of anisotropic representation, the gate weight of four direction branches is dynamically calculated according to the description vector and language embedding, the dynamic weighting of fusion features is realized, and the visual feature embedding is obtained by mapping the fusion features to the decoder embedding space;Autoregressive multilingual decoder, by adding separable two-dimensional position coding in visual feature embedding and flattening as sequence as visual prefix, combined with task prompt word containing language identifier, task type, output format and optional spatial prompt, to generate output.

Description

Technical Field

[0001] This invention belongs to the field of document parsing, and in particular relates to a multimodal adaptation device for low-resource, multilingual document parsing. Background Technology

[0002] With the rapid development of low-resource multilingual online content, the emergence of archives and educational materials in minority languages, as well as online forms, legal documents, and cultural knowledge bases, has made document parsing a core challenge for content analysis and knowledge accessibility. Furthermore, a large portion of online digital resources remains unstructured, visually encoded, and poorly digitized, especially in the low-resource multilingual category. Therefore, enabling machines to understand and parse this part of multilingual online documents is crucial for building inclusive information systems and promoting equitable access to global knowledge. However, parsing low-resource multilingual online documents still presents significant challenges. First, the layout of online documents is highly diverse, generally including multi-column text, tables, embedded images, and vertical or bidirectional writing formats rendered from HTML / CSS. Second, the diversity of languages ​​across languages ​​(e.g., Latin, Cyrillic, Sanskrit, and Chinese-Japanese-Korean) leads to severe representational imbalances, particularly for resource-scarce languages ​​such as Tibetan, Mongolian, and Uyghur, whose fonts, spacing, and writing direction differ significantly from mainstream multilingual languages. Third, in low-resource and multilingual environments, the lack of annotated data and standardized benchmarks severely limits model training and fair evaluation. Therefore, current online text content remains linguistically fragmented, with marginalized languages ​​often excluded from automated analysis pipelines. Furthermore, existing document parsing methods, such as Dolphin, mPLUG-DocOwl, and Qwen2.5-VL, are primarily trained on high-resource, multilingual data, limiting their generalization to low-resource, multilingual document parsing.

[0003] Existing technical solutions include:

[0004] (1) Piping method

[0005] Early research on document parsing primarily followed a modular pipeline design, breaking down the parsing process into multiple independent, sequentially executed subtasks. A typical approach involved performing layout detection to identify paragraphs, tables, formulas, and images within the document, then calling specialized recognizers such as OCR engines or table structure recovery models, and finally fusing some of the output into a structured result.

[0006] (2) End-to-end method

[0007] Driven by large-scale model pre-training, end-to-end methods have become the mainstream paradigm. General Visual Language Models (VLMs), including GPT-4V, InternVL, DeepSeek-VL2, GLM4v-plus, MiniCPM-o, and Qwen-VL, have demonstrated strong zero-shot and cross-task capabilities by learning from large image-text corpora. The core of end-to-end methods in document parsing is to use a single deep learning model to process the raw document into a structured target document, eliminating the need for manual breakdown of intermediate steps, and relying on a large amount of labeled data for training.

[0008] (3) Large model specifically for documents

[0009] For example, Donut, the LayoutLM series, UDOP, UniDoc, and Monkey explicitly integrate multimodal pre-training, encoder-decoder structures, and layout priors to capture complex document semantics. Recent large-scale models, including GOT, Fox, Vary, mPLUG-DocOwl, Ocean-OCR, wuukong-reader, and SmolDocling, all aim for unified document modeling across multiple element types. The core of existing solutions is to utilize "feature fusion from multiple document modalities and pre-training with domain-specific data" to achieve high-accuracy document parsing. They demonstrate significant improvement in parsing accuracy within a fixed domain and can handle complex documents with multiple elements.

[0010] Problems with existing technology:

[0011] (1) Piping method

[0012] Even pipeline-based systems like MinerU are susceptible to errors propagating between modules and a lack of global consistency, especially when dealing with complex, cross-page, or rendered documents. Furthermore, these pipeline-based systems are often designed for development and benchmarking in resource-intensive mainstream languages, thus severely limiting their generalization to scripts with different visual characteristics or writing styles.

[0013] (2) End-to-end method

[0014] Although the aforementioned general visual language models (VLMs), including GPT-4V, InternVL, DeepSeek-VL2, GLM4v-plus, MiniCPM-o, and Qwen-VL, have demonstrated strong zero-shot and cross-task capabilities, these models are primarily optimized for natural image understanding or open-domain inference, rather than for dense text layouts or structured elements. This results in unstable parsing accuracy for document-oriented tasks.

[0015] (3) Large model specifically for documents

[0016] While large models specifically designed for document parsing have developed rapidly and made great progress, most of these existing dedicated large models are English-centric or trained on datasets of mainstream languages ​​with high resources. Their effectiveness drops sharply in resource-scarce scenarios, especially for languages ​​with complex scripts (such as Tibetan, Uyghur, and Mongolian), highlighting the key gap in multilingual generalization and cross-linguistic adaptability.

[0017] As can be seen from the background technology, current web document parsing technologies still have some problems. First, in low-resource languages, most data resources are unstructured, based on visual encoding, or have a low degree of digitization. Second, web document layouts are highly diverse, and cross-lingual textual diversity leads to representational imbalances; low-resource languages ​​lack labeled data and standardized benchmarks. Finally, existing document parsing systems are mainly developed for high-resource languages ​​and generally cannot generalize to low-resource language scenarios. Furthermore, these systems are primarily trained on high-resource language and visually homogeneous data, which limits their generalization to rendering multilingual documents with different scripts and structures. In addition, most multilingual VLMs rely on large-scale text pre-training without layout or spatial foundations, resulting in poor structure recovery and unstable performance in low-resource scenarios. Therefore, there is a need for a unified framework that explicitly integrates visual, linguistic, and structural learning to effectively handle low-resource and web-based multilingual documents. Summary of the Invention

[0018] To address the above technical problems, this invention proposes a multimodal adaptation device for low-resource, multilingual document parsing. The specific technical solution is as follows:

[0019] A multimodal adaptation device for low-resource, multilingual document parsing, comprising:

[0020] The visual encoder performs hierarchical encoding of images through progressive block merging, generates a set of feature maps, and performs projection and spatial alignment to generate visual feature maps with a unified spatial grid. At the same time, it extracts visual context vectors describing the page level through global average pooling.

[0021] The adaptive vision-text mapping module decomposes the visual feature map into four anisotropic representations: horizontal, vertical, diagonal, and global. It dynamically calculates the gating weights of the four directional branches based on the visual context vector and language embedding, realizing dynamic weighting of the fused features. The fused features are then mapped to the decoder embedding space to obtain the visual feature embedding.

[0022] An autoregressive multilingual decoder generates structured output by incorporating separable two-dimensional positional encoding into visual feature embeddings and flattening it into a sequence as a visual prefix, combined with task cue words that include language identifiers, task type, output format, and optional spatial cues.

[0023] This invention offers the following advantages: Compared to existing technologies, by introducing a direction-aware, vision-to-text adaptive modeling mechanism and a script-conditional parameter sharing strategy, it effectively solves the problem of unstable parsing of low-resource multilingual documents under conditions of diverse writing directions, complex layout structures, and scarce annotation data. This method can explicitly model the spatial structural features of different writing systems while maintaining a controllable model size, and achieves collaborative optimization between overall document structure understanding and local text parsing. This significantly reduces common parsing errors such as structural inconsistencies, reading order errors, and region merging, improving the accuracy, robustness, and generalization ability of parsing complex multilingual documents. This invention has broad application prospects and significant social and economic value in fields such as the digitization of ethnic minority documents, the organization of historical archives, the processing of multilingual government documents, cross-language knowledge services, and the protection of cultural heritage. Attached Figure Description

[0024] Figure 1 This is a frame diagram of the device of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above objectives, this invention adopts the following technical solution.

[0026] This invention is a modular framework designed to resolve the dual contradiction between the scarcity of low-resource data and the heterogeneity of multilingual texts. This framework coordinates macro-structure detection and micro-content parsing within a unified architecture, providing a robust, structured understanding solution for documents with diverse writing styles and complex spatial organization. Figure 1As shown, the framework consists of three collaborative modules: (1) A visual encoder extracts high-resolution, layout-preserving feature maps to maintain the structural integrity of the document image. (2) An adaptive visual-text mapping module connects different modalities through four parallel directional branches (horizontal, vertical, diagonal, and global). This module uses a language-conditional gating mechanism to dynamically weight these branches and uses language-conditional projection to achieve parameter sharing within the language family, thereby maximizing learning efficiency in the case of scarce data. (3) An autoregressive multilingual decoder uses a cue word scheme to translate the aligned features into a task-specific structured format (such as JSON, HTML).

[0027] This invention employs the Swing Transformer as a visual encoder. Its hierarchical architecture and sliding window mechanism are particularly suitable for such tasks because they can effectively capture long-range layout dependencies (such as multi-column structures) and fine-grained local text strokes that are crucial for low-resource text recognition.

[0028] Given an image , Image height, Given the image width, the visual encoder divides it into segments of size . Non-overlapping tiles are used to form an initial tile grid with a height of [value missing]. Width is Subsequently, the Swin Transformer performs hierarchical encoding on these blocks through a four-stage progressive block merging process, generating a set of feature maps. Each stage reduces the spatial resolution by a factor of two. The output of the last stage is then used. In the standard four-phase Swin architecture, , , For feature dimensions. To ensure that subsequent adaptive modules have a uniform spatial grid, 1×1 convolution and bilinear interpolation are used to... Projection, spatial alignment to a predefined resolution (For example ):

[0029] ;

[0030] Where Interpolate is a bilinear interpolation function. These are the visual features after alignment. It is a 1×1 convolution. This represents the total number of visual lexical units. This fixed grid representation ensures that the directional branches in the visual-text mapping module operate in a stable spatial coordinate system.

[0031] In addition, to provide a global context for subsequent gating mechanisms, visual context vectors of page-level descriptions are extracted using Global Average Pooling (GAP). :

[0032] ;

[0033] generated It encapsulates the overall document style and text distribution, which are used as conditional priors to adjust the directional branches in the adaptive visual-text module.

[0034] In low-resource environments, multilingual documents exhibit high structural heterogeneity: multi-column layouts and tables introduce complex long-range dependencies, while different languages ​​display drastically different dominant stroke directions (e.g., horizontally arranged Latin, vertically arranged Mongolian, or Brahmi scripts with numerous diacritics). To capture these spatial and linguistic priors without causing drastic parameter inflation, this invention proposes an "adaptive visual-text mapping module." This module achieves reconciliation between visual features and linguistic context through directional decoupling and linguistic conditional projection.

[0035] Directional filtering: In order to explicitly model diverse reading orders and stroke patterns, visual feature maps are filtered... It is decomposed into four anisotropic representations. Four parallel convolutional branches are applied. Each branch aims to preserve spatial dimensions while specifically extracting particular spatial priors:

[0036] ;

[0037] in, Convolution functions in a specific direction, Branch features, also known as feature representations obtained after filtering in a specific direction, are branches. It includes horizontal, vertical, diagonal, and global spatial patterns. Specifically, it uses depthwise separable 1×k convolutions (horizontal) to capture the continuity of line-oriented text; k×1 convolutions (vertical) to accommodate vertically formatted text; 3×3 dilated convolutions (diagonal) with a dilation rate of r to capture slanted strokes and complex table boundaries; and a global branch consisting of 1×1 convolutions, global average pooling, and broadcasting back to H×W. This architecture ensures full coverage of the main spatial patterns while maintaining model simplicity.

[0038] Adaptive gating: To dynamically adjust the contributions of branches in different directions, a multimodal gating mechanism is introduced. Gating weights. It also depends on the visual context vector extracted by the image encoder. and language embedding To support tasks of different granularities, such as global layout detection and local script parsing, this gating mechanism is functionally divided into two modes: (1) Agnostic mode: When the specific language identity is unknown or irrelevant (e.g., during initial layout positioning), a general language embedding is used. This forces gating to take precedence. (2) Specific patterns: When parsing fine-grained content, gating is performed using the target language identity. As a condition, activate a special filter for a specific stroke pattern.

[0039] Gating weights The calculation method is as follows:

[0040] ;

[0041] in, This is the gating weight vector, a vector containing four values, each corresponding to one of the weights in one of the four directional branches. Softmax is the normalization function, ensuring that the sum of the four weights equals 1, forming a probability distribution. It is a learnable language embedding. This indicates a splicing operation. Mapping visual features to the language space , These are the parameters of the gating layer. The former is a learnable weight matrix, responsible for mapping the concatenated mixed information to a 4-dimensional space, while the latter is a bias term, a standard learnable constant in the neural network layer. The result represents a 4-dimensional vector. This leads to... This represents the normalized attention weights on the four directional branches. The fused features are:

[0042] ;

[0043] in, It involves fusing features from the four directional branches; all other parameters are defined earlier.

[0044] This dual-conditional adjustment enables the adapter to respond sensitively to layout changes (e.g., enhancing vertical features for Mongolian pages) and is robust to mixed content common in low-resource documents.

[0045] Script-based conditional projection: This invention uses a script-shared 1×1 projection to fuse features. Mapping to the decoder embedding space:

[0046] ;

[0047] Among them, parameters It is a mapping function , It is a mapped embedding. It is language script families (Latin, Sanskrit, etc.). The matching decoder conforms to the embedding dimension. With low resources, sharing parameters within these language families allows the model to leverage cross-linguistic visual homology (e.g., shared stroke shapes and ligatures), significantly improving sample efficiency and generalization ability to underrepresented languages.

[0048] The autoregressive multilingual decoder is specifically built upon Qwen 2.5-Instruct, a model with powerful multilingual generation and instruction following capabilities. To improve its performance on low-resource languages, it was incrementally trained using monolingual corpora of 11 underrepresented languages. This adapted model is called ML-Qwen.

[0049] To connect visual features with an autoregressive multilingual decoder while preserving spatial location cues, in the embedding Separable 2D position encoding has been added. The generated tensor is then flattened into a sequence, which serves as a visual prefix:

[0050] ;

[0051] in, This represents the flattened sequence. To represent flattening, each visual word is represented by its row / column position. Anchoring enables the decoder to reason about complex layout topologies, such as reading order and table cell alignment, rather than just handling flattened indexes. This approach ensures compatibility with standard autoregressive decoding while maintaining parametric efficiency.

[0052] The autoregressive multilingual decoder is guided by a concise task cue that coordinates the processing stages and output requirements:

[0053] ;

[0054] in, The task is defined as page_structure, which performs macro-level structure detection. Other tasks perform micro-level content parsing, including text line recognition, paragraph recognition, table recognition, and formula recognition. It is a language identifier; for language-independent page_structure tasks, Set to Univ; for other tasks, It is a specific language identifier.

[0055] and This indicates the output format, which can be selected from JSON, Markdown, and HTML.

[0056] [Box] is a spatial hint; for region-level resolution, you can choose to add normalized bounding box coordinates. , This represents the x and y coordinates of a region. This is optional; you can add it or not. If you don't add it, the region can be cropped before being entered.

[0057] The final input sequence is composed of cue words and visual features:

[0058] ;

[0059] The model then produces the following output: ;

[0060] in, It is the content output by the model, such as the page structure in JSON format or text in Markdown format, etc. ML-Qwen is the decoder;

[0061] Through this design, ML-Qwen can efficiently utilize visual spatial information to achieve cross-modal understanding, from layout analysis to fine-grained content extraction, in low-resource, multilingual environments.

[0062] Model training process:

[0063] This invention employs a progressive three-stage training strategy to ensure the stability of low-resource multilingual training. The first stage adjusts the decoder's linguistic capabilities on low-resource text without visual input. The second stage performs visual feature alignment while the decoder is frozen, enabling the adapter to learn a stable cross-modal interface. The third stage performs joint fine-tuning of all components to improve global consistency and the ability to generate fine-grained structures. This strategy prevents the dedicated orientation filter in the adapter from being overwhelmed by the decoder's complex autoregressive gradients during the initial alignment phase.

[0064] Phase 1: Continuous Language Modeling. Based on monolingual corpora and cross-lingual corpora of 11 underrepresented languages, we will continue to carry out causal language modeling (CLM) work to improve lexical fluency and the efficiency of using structured lexical units.

[0065] ;

[0066] in, Represents the language model loss. This represents the probability distribution predicted by the model. These are model parameters. This represents the current target word, specifically the word at position i in the sequence. This represents all the tokens in the sequence that precede position i. The formula means: given the words that have already appeared... The computational model predicts the next correct word. The probability is calculated and its negative logarithm is used as a penalty. If the model predicts accurately, the loss is small. LoRA (low-rank adaptive) technique with rank r=8 is used on the Qwen module, while keeping the basic weights frozen to preserve the model's inherent multilingual knowledge.

[0067] Phase Two: Visual-Language Alignment. Subsequently, the decoder is frozen, and the visual encoder and adaptive visual-text mapping module are trained together. This phase forces the adapter to project visual evidence into a latent space compatible with the decoder without interfering with established language priors.

[0068] Given an input sequence and the real label sequence Minimize cross-entropy loss:

[0069] ;

[0070] in, This represents the alignment loss, specifically used in the second stage of training, measuring the difference between the structured text (such as JSON or layout information) generated by the model and the true labels. This represents the total length of the target text sequence. It is the log probability predicted by the model. It is a probability distribution. This represents the set of model parameters. The target word in the current step, i.e., the word the model is working on. The correct character or word that should be predicted at any given moment. This represents the sequence of words that the model has generated before time t. This stage aims to train the model to represent visual features in a decoder-compatible format and complete this conversion under stable language-level supervision. During this stage, Optimization is performed simultaneously in both macroscopic structure detection and fine-grained analysis tasks.

[0071] Phase Three: Joint Fine-tuning. Finally, a hybrid dataset of multimodal and plain text batches is used to jointly optimize all modules, ensuring both the accuracy of structured localization and the fluency of the text.

[0072] ;

[0073] In each training step, multimodal batches are used Optimize the LoRA adapters for the visual encoder and decoder, while using alternating plain text batches. Update the same LoRA adapter. The base decoder weights remain fixed to maintain multilingual fluency while achieving lightweight adaptation via LoRA. This invention sets... To balance visual fundamentals and text fluency.

[0074] Model reasoning process:

[0075] This invention employs a recursive parsing protocol from macro to micro levels, decoupling global layout understanding from local text decoding. This strategy, through a two-stage execution process, ensures the model's robustness in multilingual environments.

[0076] Phase 1: Macro-structure Detection. First, the complete document image is processed using language-independent cue words and the `page_structure` task. The model identifies semantic entities and generates structural information in JSON format, including bounding boxes, categories, and reading order. This phase focuses on spatial topology and layout organization, unaffected by the specific language of the document.

[0077] Phase Two: Fine-grained Content Parsing. For each detected region, the model performs local parsing. Guided by language-specific cues, the region is located using normalized coordinates or local cropping. During this phase, a gating mechanism activates a directional filter specifically tailored to the language of that block.

[0078] This decoupled reasoning strategy enables the method to balance the scalability of the layout with the accuracy of recognition, effectively solving the common "structure-language interweaving" problem in complex multilingual documents.

Claims

1. A multimodal adaptation device for low-resource, multilingual document parsing, characterized in that, include: The visual encoder performs hierarchical encoding of images through progressive block merging, generates a set of feature maps, and performs projection and spatial alignment to generate visual feature maps with a unified spatial grid. At the same time, it extracts visual context vectors describing the page level through global average pooling. The adaptive vision-text mapping module decomposes the visual feature map into four anisotropic representations: horizontal, vertical, diagonal, and global. It dynamically calculates the gating weights of the four directional branches based on the visual context vector and language embedding, realizing dynamic weighting of the fused features. The fused features are then mapped to the decoder embedding space to obtain the visual feature embedding. An autoregressive multilingual decoder generates structured output by incorporating separable two-dimensional positional encoding into visual feature embeddings and flattening it into a sequence as a visual prefix, combined with task cue words that include language identifiers, task type, output format, and optional spatial cues.

2. The multimodal adaptation device for low-resource, multilingual document parsing according to claim 1, characterized in that, In a visual encoder, the image is divided into segments of size 1. Non-overlapping tiles are used to form an initial tile grid. These tiles are then hierarchically encoded through a four-stage progressive tile merging process, resulting in a set of feature maps. Take the output of the last stage. Projection, spatial alignment to a predefined resolution: ； Where Interpolate is a bilinear interpolation function. It is the aligned visual feature map. It is a 1×1 convolution. For image height, width, and length; Visual context vectors for page-level descriptions are extracted using Global Average Pooling (GAP). : 。 3. The multimodal adaptation device for low-resource, multilingual document parsing according to claim 1, characterized in that, In the adaptive vision-text mapping module, visual feature maps are... It is decomposed into four anisotropic representations, and four parallel convolutional branches are used. Each branch extracts specific spatial priors while preserving spatial dimensions: ； in, For a specific direction of convolution function, The branch is the feature representation obtained after filtering in a specific direction. Includes: horizontal, vertical, diagonal, and global.

4. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 3, characterized in that, In the adaptive vision-text mapping module, a multimodal gating mechanism is introduced to dynamically adjust the contributions of branches in different directions, and the gating weights... It also depends on the visual context vector extracted by the image encoder. and language embedding Gating weights The calculation method is as follows: ； in, This is a gated weight vector, a vector containing four values, each corresponding to a weight in one of the four directional branches. Softmax is a normalization function that ensures the sum of the four weights equals 1, forming a probability distribution. It is a learnable language embedding. This indicates a splicing operation. Mapping visual features to the language space , These are the parameters of the gating layer. The former is a learnable weight matrix, responsible for mapping the concatenated mixed information to a 4-dimensional space, and the latter is the bias term. The result represents a 4-dimensional vector, from which... The normalized attention weights represent the four directional branches; the fused features are: ； in, It involves integrating features from the four directional branches.

5. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 4, characterized in that, In the adaptive vision-text mapping module, a 1×1 projection shared by the script is used to fuse features. Mapping to the decoder embedding space: ； Among them, parameters It is a mapping function , It is a mapped embedding. It is language script family, The matching decoder conforms to the embedding dimension.

6. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 1, characterized in that, The autoregressive multilingual decoder specifically refers to: in the embedding Add separable 2D position encoding Then, the generated tensor is flattened into a sequence, which is used as a visual prefix: ； in, This represents the flattened sequence. Indicates flattening; The autoregressive multilingual decoder is guided by a task cue word. Coordination and processing phases and output requirements: ； in, Indicates the task. It is a language identifier. Indicates the output format; [Box] is a space hint. The final input sequence is composed of cue words and visual features: ； The model then produces the following output: ; in, ML-Qwen is the output of the model and is the decoder.

7. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 1, characterized in that, A progressive three-stage training strategy is adopted. In the first stage, the language ability of the decoder on low-resource text is adjusted without visual input. In the second stage, visual feature alignment is performed while the decoder is frozen, so that the adapter learns a stable cross-modal interface. The third stage involves joint fine-tuning of all components to improve global consistency and the ability to generate fine-grained structures.

8. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 7, characterized in that, In Phase One, ； in, Represents the language model loss. This represents the probability distribution predicted by the model. These are model parameters. This represents the current target word, specifically the word at position i in the sequence. This represents all the tokens in the sequence that are located before position i.

9. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 7, characterized in that, In Phase Two, Given an input sequence and the real label sequence Minimize cross-entropy loss: ； in, Indicates alignment loss. Represents the total length of the target text sequence. It is the log probability predicted by the model. It is a probability distribution. Represents the set of model parameters. The target word in the current step, i.e., the word the model is working on. The correct character or word that should be predicted at any given moment. This represents the sequence of words that the model has generated before time t. .

10. A multimodal adaptation device for low-resource, multilingual document parsing according to claim 7, characterized in that, In Phase 3, a hybrid dataset of multimodal and plain text batches is used to jointly optimize all modules, ensuring both the accuracy of structured localization and the fluency of the text. ； These are the weighting coefficients.

Images