Method and system for generating high-fidelity multilingual ancient book annotation driven by reverse synthesis

Abstract

The application provides a multi-lingual ancient book high-fidelity marking generation method and system driven by reverse synthesis, and belongs to the technical field of document image processing, and comprises the following steps: S1, constructing an OCR corpus covering multiple languages; S2, based on the writing rules specific to each language, using a multi-lingual structure modeling algorithm to perform layout modeling and visual rendering; S3, constructing a degradation model for degraded samples, and precisely adjusting the degradation type, intensity and distribution through multi-parameter mapping and combination control; S4, automatically evaluating the quality of the degraded samples, and dynamically self-correcting and closed-loop optimizing according to the evaluation results to obtain the final degraded samples; S5, using a lightweight target detection and OCR identification network to perform semantic-level marking on the final degraded samples to generate the final standardized marking results. The method provides a solid technical foundation for constructing a high-quality, multi-lingual historical literature database.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of document image processing, and particularly relates to a multi-lingual ancient book high-fidelity labeling generation method and system driven by reverse synthesis. BACKGROUND

[0002] Early digitalization of ancient books mainly adopts heuristic rule methods such as page template matching, edge detection and region segmentation, and the text area and image area are segmented by manually setting page structure rules. Such methods perform well in standardized printed documents, but when faced with irregular layout, misaligned text, overlapping annotations, paper deformation and other situations in ancient books, the rules are easily invalid, and it is difficult to realize large-scale automatic processing. Representative works include the Chinese Reborn Ancient Book Digitalization Project of the National Library of China and some page structure analysis methods based on template matching.

[0003] With the development of computer vision, researchers began to use convolutional neural networks (CNN), fully convolutional networks (FCN) and object detection models (such as U-Net, YOLO, Transformer OCR) for document recognition and repair tasks. Typical methods include image enhancement, denoising, binarization, distortion correction and shadow removal. Such methods can improve the readability of ancient books at the visual level, but are limited by local receptive fields and are difficult to capture long-range layout relationships and character structures in complex backgrounds. For example, the DiffHDR model proposed by Yang et al. uses a diffusion network to achieve image-level repair of damaged ancient books, and improves repair consistency by introducing semantic and spatial conditions; but its training relies on a large number of manually annotated samples and is mainly aimed at single-lingual historical documents.

[0004] To overcome the limitations of single-modal repair, researchers propose a multi-stage scheme combining OCR, language models and generative networks. The AutoHDR framework proposes a three-stage fusion process: ① OCR-assisted damage localization (OADL); ② language model-driven content prediction (DCP); ③ image generative appearance reconstruction (HAR). This method significantly improves the automation of repair, and in the case of severe damage, the recognition rate is improved from 46.83% to 84.05%. However, its training still relies on manually collected and true repair samples, with high data acquisition costs, and limited generalization ability in multi-lingual environments.

[0005] In recent years, researchers have begun to explore synthetic data-driven OCR training paradigms. For example, Deep Aramaic (ICDAR 2025) proposed by Ding et al. generates high-fidelity inscription data through three-dimensional glyph modeling and weathering simulation; SARD Dataset developed by Ahmed et al. (arXiv:2505.24600) generates large-scale Arabic OCR samples using programmatic typesetting. Although these methods have made progress in single-language documents, they have failed to simulate the physical degradation characteristics and multi-element layout structure of ancient books, and lack automated annotation mechanisms, limiting their application in complex ancient book scenarios.

[0006] However, the above prior art has the following defects:

[0007] (1) Data scarcity and lack of realism: existing ancient book recognition and restoration models rely on manually scanned and annotated data, and the available samples of ancient books in languages such as Mongolian, Tibetan, and Manchu are extremely limited, making it difficult for the model to generalize to real-world scenarios. Although synthetic data methods can supplement samples, they often lack physical characteristics such as paper texture, ink spread, and aging light and shadow, resulting in a significant gap in distribution between generated samples and real ancient books.

[0008] (2) Insufficient modeling of language structure: Most synthetic methods are designed for modern printed texts or single-language documents, with simple typesetting rules that cannot handle Mongolian vertical writing, Tibetan complex stacking, and Manchu horizontal symbol systems. The lack of multi-language script structure results in distorted generated data in terms of character morphology and page layout, making it difficult to meet the training needs of multi-language OCR models.

[0009] (3) Lack of automated annotation and controllable degradation mechanism: Traditional synthetic samples usually only generate image layer information without providing structural annotations such as characters, lines, notes, and seals. The degradation process relies on random blurring or noise addition, which cannot accurately control the type and intensity of damage, making it difficult to use the data for structured training and evaluation.

[0010] (4) Fragmented processing flow, insufficient scalability: Existing document generation, restoration, and recognition technologies are mostly independent modules, lacking a unified automated pipeline. They cannot achieve continuous processing from content generation to structure annotation, and lack language adaptation capabilities and scalable data fusion mechanisms. SUMMARY

[0011] To solve the above technical problems, the present application provides a reverse synthetic-driven multi-language ancient book high-fidelity annotation generation method, comprising the following steps:

[0012] Step S1: Collect multi-language ancient book text data from multiple sources; through language tokenization, noise cleaning, and structure standardization processing, build an OCR corpus covering multiple languages;

[0013] Step S2: Based on the writing rules specific to each language, use a multilingual structure modeling algorithm for typesetting modeling and visual rendering, and realize preliminary structure-level labeling according to the spatial hierarchical relationship of the text, title, annotation, seal, illustration;

[0014] Step S3: Construct a degradation model for the degradation sample, realize precise adjustment of the degradation type, intensity and distribution through multi-parameter mapping and combination control, so that the degradation sample is highly consistent with the real ancient books in terms of texture, brightness and structure level;

[0015] Step S4: Design a multi-dimensional quality evaluation and reverse feedback mechanism, automatically evaluate the quality of the degradation sample, and dynamically self-correct and closed-loop optimize according to the evaluation results to obtain the final degradation sample;

[0016] Step S5: Use a lightweight target detection and OCR recognition network to perform semantic-level labeling on the final degradation sample, combine read-back consistency detection, region IOU correction and multi-model consistency determination to generate the final standardized labeling result.

[0017] Advantages:

[0018] 1. The application provides a multilingual ancient book high-fidelity labeling generation method driven by reverse synthesis, which starts from the visual attributes of real ancient book samples, controls the generation process through parameter backtracking, forms a bottom-up and closed-loop reverse generation path.

[0019] 2. The application establishes a parameterized typesetting template for language system differences such as Mongolian, Tibetan and Manchu, and realizes automatic language conversion.

[0020] 3. The application simulates the paper and ink degradation process through a physical parameterized model, generates images that are highly similar to real ancient books in visual distribution, and combines PSNR, SSIM, CLIPScore, OCR Accuracy, Structural IoU and other indicators to complete quality feedback.

[0021] 4. The application realizes synchronous output of synthesis and labeling through a target detection network and an OCR recognition network, significantly reducing manual intervention.

[0022] 5. The application supports multilingual, mixed layout, and multi-task output, and has cross-domain adaptability and data reuse capability.

[0023] 6、The reverse synthesis method of the application takes the visual and physical feature distribution of real ancient books as the starting point, realizes reverse modeling from real samples to synthetic data through parameter backtracking, forms a closed-loop control path of "real sample feature distribution parameter backtracking controllable generation quality feedback automatic labeling", and provides unified data support for multilingual ancient book identification, digital protection and intelligent utilization. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 A reverse synthesis driven multilingual ancient book high-fidelity labeling generation method flowchart of the application;

[0025] Figure 2 A structure block diagram of a reverse synthesis driven multilingual ancient book high-fidelity labeling generation system of the application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the application more clear, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and not to limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0027] Example 1

[0028] As shown in the figure, the reverse synthesis driven multilingual ancient book high-fidelity labeling generation method provided by the embodiment of the application comprises the following steps: Figure 1 Step S1: Collect multilingual ancient book text data from multiple sources; through language word segmentation, noise cleaning and structure standardization processing, build an OCR corpus covering multiple languages;

[0029] Step S2: Based on the writing rules specific to each language, use a multilingual structure modeling algorithm for layout modeling and visual rendering, and realize preliminary structure level labeling according to the spatial hierarchical relationship of the main text, title, annotations, seals and illustrations;

[0030] Step S3: Construct a degradation model for degraded samples, realize precise adjustment of degradation type, intensity and distribution through multi-parameter mapping and combination control, so that the degraded samples are highly consistent with real ancient books in terms of texture, brightness and structure;

[0031] Step S4: Design a multi-dimensional quality evaluation and reverse feedback mechanism, automatically evaluate the quality of the degraded samples, and dynamically self-correct and closed-loop optimize according to the evaluation results to obtain the final degraded samples;

[0032]

[0033] ​Step S5: Use a lightweight object detection and OCR recognition network to perform semantic-level annotation on the final degraded samples, and combine readback consistency detection, region IOU correction and multi-model consistency judgment to generate the final standardized annotation results.

[0034] In one embodiment, step S1 above: collecting multilingual ancient text data from multiple sources; constructing an OCR corpus covering multiple languages ​​through language segmentation, noise removal, and structural standardization, specifically including:

[0035] This invention collects ancient text data in Mongolian, Tibetan, Manchu, and other languages ​​from multiple sources, including open corpora, OCR-extracted text, manually entered materials, and customized documents. Through language segmentation, noise removal, and structural standardization, a multilingual OCR corpus covering various languages ​​is constructed. This invention supports multiple methods such as automatic crawling, manual input, and file import, ensuring the diversity of corpus sources and the authenticity of content, providing a reliable semantic foundation for subsequent data generation. Unlike traditional methods, the corpus collected in this step not only serves text rendering but is also used for subsequent alignment with the feature distribution of real samples, achieving integrated semantic and visual generation preparation.

[0036] In one embodiment, step S2 above involves using a multilingual structural modeling algorithm to perform typesetting modeling and visual rendering based on the unique writing rules of each language, and to achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations. Specifically, this includes:

[0037] Step S21: Design a basic template to provide a general framework for multilingual typesetting.

[0038] In the process of digitizing and typesetting ancient books in multiple languages, traditional typesetting methods struggle to uniformly address the typesetting needs of different languages ​​due to significant differences in writing direction, typesetting order, and character structure among languages ​​such as Mongolian, Tibetan, and Manchu. Therefore, to ensure the system can efficiently and consistently generate typesettings for multilingual ancient books, this invention designs a basic template that provides a universal framework and flexible extensibility for multilingual typesetting.

[0039] The basic template is a standardized set of parameters designed to accommodate multilingual typesetting needs. Its main goal is to provide a unified typesetting framework to ensure the accuracy, scalability, and consistency of multilingual ancient book typesetting. This template allows for maintaining visual consistency while flexibly addressing the specific typesetting requirements of each language when handling multiple languages.

[0040] (1) Core components of the basic template:

[0041] 1. Layout rules: including basic layout specifications such as writing direction (vertical or horizontal), line priority or column priority.

[0042] 2. Visual parameters: such as font size, kerning, line spacing, etc., to ensure the readability and consistency of the text.

[0043] 3. Structure definition: defining structural elements on the page (such as text, captions, annotations, etc.), ensuring the rationality of the layout structure.

[0044] (2) The role of the basic template:

[0045] 1. Unified framework: providing a common parameter setting for multi-lingual layout, ensuring the uniformity of layout effects for different languages.

[0046] 2. Automatic adaptability: the system can automatically adapt to new languages based on the basic template, adjusting the difference items (such as writing direction, character density, etc.) to generate a layout template that meets the requirements of the new language.

[0047] 3. Simplify design: avoid starting from scratch to design layout rules, quickly generate a new language layout format by adjusting the difference items with the basic template.

[0048] 4. Support for subsequent modeling: the basic template provides the necessary layout framework for structural modeling and automatic annotation, ensuring accurate execution of subsequent steps.

[0049] Step S22: Construct a unified intermediate layout representation layer ILR to abstractly describe the spatial structure and writing properties of different language ancient book pages.

[0050] To achieve unified layout and structure mapping of Mongolian, Tibetan, and Manchu ancient books of multiple languages, the present invention designs a multi-lingual structure modeling algorithm in the rendering engine. This algorithm takes the "Intermediate Layout Representation (ILR)" as the core, and through the unified abstraction and parameterized modeling of the layout rules between scripts, it realizes the computable expression and extensibility generation of language differences.

[0051] The present invention first constructs a unified intermediate layout representation layer ILR to abstractly describe the spatial structure and writing properties of different language ancient book pages. ILR is a cross-language shared semantic-geometric intermediate layer, and its data structure is shown in Table 1.

[0052] Table 1 ILR data structure definition table

[0053]

[0054]

[0055] Table 1 defines the core fields and parameter ranges of the intermediate layout representation layer (ILR). ILR unifies the layout elements of different language pages in a structured manner, and the logical relationship between its fields is as follows:

[0056] 1. script_type determines the structure mapping rule called;

[0057] 2. writing_direction and baseline_orientation jointly control the text direction and layout direction;

[0058] 3. line_spacing, char_spacing and char_density jointly reflect the writing density;

[0059] 4. anchor_graph establishes the spatial hierarchy between different functional areas of the page, which is a key interface for the subsequent automatic labeling stage. anchor_graph describes the hierarchy and adjacency relationship of areas such as text, captions, annotations, illustrations and seals in the form of nodes (V) and edges (E), which is an important structural bridge connecting subsequent automatic labeling modules. As a unified interface for layout, ILR takes on the statistical characteristics of real ancient books and drives layout generation and labeling output, achieving language-independent structural expression.

[0060] Step S23: Based on the differences in writing systems of Mongolian, Tibetan and Manchu languages, corresponding structure mapping rules and parameterized controls are defined to map ILR parameters to language-specific template parameter sets.

[0061] To adapt to the differences in writing systems of the three main languages of Mongolian, Tibetan and Manchu, the invention defines corresponding structure mapping rules and parameterization control tables as shown in Table 2.

[0062] Table 2 Structure mapping rules and parameter table

[0063]

[0064] In the generation process, according to script_type, the corresponding rule set is automatically called to map ILR parameters to language-specific template parameter set Θ s , forming a unified structure to rendering control path. The mapping function is defined as follows:

[0065] Φ s : ILR→ Θ s , Θ s = { θ grid , θ spacing , θ glyph , θ punct , θ anchor}.

[0066] wherein: θ grid is column grid and paragraph block layout; θ spacing is line spacing, word spacing and column spacing control; θ glyph is glyph aspect ratio, font size variation; θ punct is punctuation rotation and alignment strategy; θ anchor is structural block anchor and hierarchy.

[0067] Step S24: design a template parameter dynamic expansion mechanism to support quick access and automatic adaptation of new languages, when a new language is identified, only the difference items between the new language and the basic template need to be supplemented, and the template parameter set of the language is automatically generated by an adaptive function.

[0068] To enhance universality, the application proposes a template parameter dynamic expansion mechanism to support quick access and automatic adaptation of new languages. When a new language s new is identified, only the difference items Δ script (including writing direction, baseline direction, character density and other key parameters) between the new language and the basic template need to be supplemented, and the typesetting template parameter set Θ

[0069] P new = f adapt (ILR base , Δ script );

[0070] wherein, f adapt is an adaptive function; ILR base is the ILR of the basic template; P new is the template of the new language;

[0071] the typesetting template parameter set Θ new of the language is automatically generated. This mechanism enables the method to have the generalization ability of "language as template", and can support the typesetting generation of new or mixed scripts without manual adjustment. The expansion logic is as follows:

[0072] 1. If it is vertical text, the system preferentially enables column-first layout;

[0073] 2. If char_density is high, automatically reduce line_spacing;

[0074] 3. If the new script has diacritic or ligature features, enable ligature_mode or diacritic_policy parameter group.

[0075] This mechanism significantly improves the portability and universality in the tasks of new language, mixed script or cross-domain ancient book typesetting, and guarantees the consistency and automation degree of page generation.

[0076] The layout parameters of this step are not manually preset, but are dynamically driven by the statistical distribution generated by the parameter backtracking of real sample features. The rendering engine introduces font randomization and style disturbance mechanisms during the generation process, making the generated images have rich diversity and naturalness at the visual level. The rendered images not only contain the main text, but also can superimpose elements such as annotations, captions, illustrations, and seals, thereby forming a complex and real ancient book page layout.

[0077] In one embodiment, the above step S3: constructing a degradation model for the degraded sample, through multi-parameter mapping and combination control, realizes precise adjustment of the degradation type, intensity and distribution, so that the degraded sample is highly consistent with the real ancient book in terms of texture, brightness and structure, specifically including:

[0078] Step S31: Define the degradation parameter triplet (α, β, γ), which corresponds to the three types of degradation factors of aging attenuation, light disturbance and local damage, respectively. α, β, γ are non-negative real numbers; Perform normalization operation on the parameter triplet to obtain: α' = α / (α+β+γ), β' = β / (α+β+γ), γ' = γ / (α+β+γ), where α+β+γ > 0.

[0079] The parameter range of the triplet is set as follows:

[0080] 1. Aging parameter α ∈ [0.2, 0.8]: control the time-dependent decay of paper yellowing, fiber thickening and ink penetration, etc.

[0081] 2. Light parameter β ∈ [0.1, 0.6]: adjust the light distribution, shadow direction and brightness deviation;

[0082] 3. Damage parameter γ ∈ [0, 0.3]: simulate local structural damage such as insect holes, mold spots and stains, etc.

[0083] Step S32: Construct the degradation mapping function f(·) to generate the final degraded sample:

[0084] Ideg=f(Isyn,α,β,γ)=Isyn×(1 - δ(α′A + β′L)) × (1 - γ′N) + γ′B × N;

[0085] Where, I syn is the input clear synthetic image; I degThe output is a degraded image; A, L, and N are aging mapping parameters, light mapping parameters, and noise mask parameters extracted from real ancient book samples; B is a background fill value used to simulate the true performance of the hollow area; In order to avoid image value inversion due to the amplification of the normalized coefficient, the attenuation adjustment factor δ ∈ (0, 1] is introduced, which is used to control the total amplitude of the degradation intensity and ensure that the multiplicative term is always non-negative in the physical sense; The value of δ in the embodiment is between 0.5 and 0.9. The above mapping process can also be regarded as a specific implementation of the degradation control function f(·), and the core goal is to integrate the aging, light and damage three types of degradation factors into a unified visual generation path.

[0086] In the above formula, A, L, N and B are key parameters in the degradation process, which represent different types of degradation characteristics. Specifically:

[0087] 1. AgeMap (A aging mapping parameter) is a two-dimensional matrix extracted by analyzing real ancient book samples. Each element in the matrix represents the aging degree of a pixel in the image. By modeling physical degradation phenomena such as paper aging and ink diffusion, each value in the matrix represents the aging intensity of the region. The higher the value, the more serious the aging degree of the region, which is manifested as paper yellowing, ink penetration, etc.

[0088] 2. LightMap (L light mapping parameter) is another two-dimensional matrix representing the light intensity or light disturbance of each pixel in the image. By extracting light features from real ancient book samples, LightMap simulates brightness changes, shadow effects and other phenomena caused by uneven lighting during the preservation process. Each element value in the matrix reflects the light intensity of the region in the image. Higher values indicate stronger light in the region, and lower values indicate weaker light or shadows in the region.

[0089] 3. NoiseMask (N noise mask parameter) is a binary matrix indicating whether a region in the image is subject to noise or damage (such as insect damage, mold spots, etc.). Each element value in the matrix is 0 or 1, with 1 indicating that the region has noise or damage, and 0 indicating that the region has no damage. Through NoiseMask, local damage or noise commonly found in real ancient books can be simulated, making the degradation effect more realistic.

[0090] 4、Background layer (B) is a background layer matrix with the same size as the input image Isyn (usually HxWxC, where C is the number of channels), which is used to visually fill or simulate replacement in the damaged area marked by the degradation mask N, thereby simulating the effects of paper loss, holes or structural damage, etc. The element value of B ranges from 0 to 1, representing the normalized intensity value of the image pixel. For grayscale filling scenarios, B can be a constant matrix (e.g., B=0.85); for texture filling, B can be generated by paper texture blocks extracted from real ancient book images and normalized to ensure consistency with the degraded image. Its generation method includes two paths:

[0091] (1) Paper texture is extracted from real ancient book images to construct a background texture library. During the degradation process, the system samples the corresponding size of the texture block from the damaged area location, and performs scaling and edge feathering fusion for area filling;

[0092] (2) For small-scale damaged areas, the missing area background content can be dynamically generated by edge pixel interpolation or image inpainting algorithms (such as Telea or Navier-Stokes based inpainting methods), to improve local texture continuity and overall visual naturalness.

[0093] Through these four matrices (A, L, N, B), the degradation process is precisely controlled to simulate complex degradation phenomena such as paper aging, uneven lighting and local damage. During image generation, the system integrates the effects of the three degradation factors to generate a degraded image, ensuring that the generated ancient book image is highly similar to the real one in terms of visual and physical aspects.

[0094] The present application extracts aging texture distribution, illumination decay pattern and damage noise characteristics from real ancient books, constructs multi-dimensional mapping matrices, and realizes continuous regulation of the degradation process through the following degradation control function:

[0095] Input: Clean image I_syn, degradation parameters (α, β, γ)

[0096] A←Extract(AgeMap from texture database)

[0097] L←Sample(LightMap from illumination set)

[0098] N←Generate(NoiseMask based on damage statistics)

[0099] B←Select(background fill texture)

[0100] I_deg←I_syn × (1 - δ(α′A + β′L)) × (1 - γ′N) + γ′B × N

[0101] Output: Degraded image I_deg

[0102] This step aims to simulate the natural degradation of real ancient books caused by paper aging, ink diffusion, uneven lighting, insect damage, mold stains, and other factors during long-term preservation. The core of this step is to establish a parameterized degradation model, which can precisely adjust the degradation type, intensity, and distribution through multi-parameter mapping and combination control, making the generated samples highly consistent with real ancient books in terms of texture, brightness, and structure.

[0103] The present application realizes the parameterization, controllability and quantifiability of degradation simulation, overcomes the uncontrollability and unverifiability of traditional random degradation method, and provides high-quality "clean sample-degraded sample-repair target" triplet data support for subsequent automatic labeling and OCR training.

[0104] To ensure that the generated samples meet the quantifiable high-fidelity standards in terms of visual, structural and semantic aspects, the present application designs the following multi-dimensional quality evaluation and reverse feedback mechanism. After completing the degradation simulation, automatically evaluate the quality of each batch of samples, and feed back the evaluation results to the feature modeling and degradation parameter control layer in real time, to realize dynamic self-correction and closed-loop optimization of data generation.

[0105] In one embodiment, the above step S4: designing a multi-dimensional quality evaluation and reverse feedback mechanism, automatically evaluating the quality of the degraded samples, and dynamically self-correcting and closed-loop optimizing according to the evaluation results to obtain the final degraded samples, specifically includes:

[0106] Step S41: Image fidelity index: using structural similarity SSIM and peak signal-to-noise ratio PSNR to measure the similarity of generated degraded samples and real ancient books in terms of texture, lighting and edge.

[0107] Specifically, in order to objectively quantify the visual consistency between the generated degraded ancient book samples and the real ancient book samples, the present application considers from two dimensions of pixel error statistics and human visual perception:

[0108] (1) Peak signal-to-noise ratio (PSNR)

[0109] PSNR is calculated based on Mean Squared Error (MSE) and is mainly used to evaluate the performance of generated samples in terms of illumination consistency and background noise. A higher PSNR value means that the generated degraded sample is highly consistent with the real sample in terms of pixel intensity distribution. The calculation formula is as follows:

[0110] ;

[0111] ;

[0112] wherein, and represent the generated degraded sample and the real ancient book reference image respectively, i and j represent the pixels in the image, , is the height and width of the image, is the maximum possible value of the image pixel (usually 255).

[0113] (2) Structural Similarity (SSIM)

[0114] SSIM is a perception-based metric that distinguishes from simple pixel comparison, focusing on measuring the retention of text edge sharpness and paper texture structure. The closer the SSIM value is to 1, the more realistic the generated degraded sample is in visual structure. The calculation formula is as follows:

[0115] ;

[0116] wherein, is the generated image window, is the real image window; , are the average values of , (reflecting brightness); , are the variances (reflecting contrast / texture);

[0117] is the covariance (reflecting structural similarity); , are constant values that maintain stability.

[0118] Step S42: Semantic Consistency Index: Introduce cross-modal semantic matching index CLIPScore to evaluate the matching degree of visual features and text semantic space of generated samples;

[0119] The calculation principle of semantic consistency is: using a pre-trained cross-modal representation model to map images and texts to a unified semantic space and extract their respective feature vectors; by calculating the similarity between the feature vectors, a CLIPScore score is obtained, and the higher the score, the higher the consistency between the image content and the text semantics. Among them, the text description includes page rendering text, recognized text or a combination thereof; the score can be calculated at the page level or the region level, and the scores obtained by calculation are summarized to obtain the final indicator.

[0120] Step S43: Structural annotation accuracy indicator: evaluate the region positioning and structural accuracy of automatic annotation through OCR accuracy OCR Accuracy and structural intersection over union Structural IoU.

[0121] Step S44: define a comprehensive quality score function:

[0122] Q = w1 x SSIM + w2 x PSNR + w3 x CLIPScore + w4 x OCR Accuracy + w5 x SioU;

[0123] where w1-w5 are empirical weights, satisfying ∑w i =1, i=1-5; OCR Accuracy is the OCR recognition accuracy; SioU is the structural intersection over union.

[0124] Set the minimum threshold Q min When the degradation sample is lower than the threshold, go to step S3 for parameter adjustment, specifically: trigger the automatic parameter adjustment process, including: based on Q, perform parameter search or gradient approximation update with constraints on (α, β, γ), update the mapping weights and threshold set of the degradation control function, and regenerate the degradation sample until Q≥Q min The feedback logic is as follows:

[0125] Input: Sample set S, evaluation metrics Q

[0126] If Q < Q_min:

[0127] Adjust (α, β, γ) via gradient feedback

[0128] Update feature model F'

[0129] Regenerate I_deg'

[0130] Re-evaluate Q

[0131] Else:

[0132] Save sample and export metadata

[0133] The present application realizes continuous optimization of the degradation model through gradient feedback and parameter correction, forms a closed-loop path of "feature modeling-degradation generation-quality evaluation-parameter backtracking-regeneration", and finally ensures that the degradation sample maintains stable and consistent high-fidelity features in each dimension.

[0134] In one embodiment, the above step S5: using a lightweight target detection and OCR recognition network to perform semantic-level labeling on the final degradation sample, combining read-back consistency detection, region IOU correction and multi-model consistency judgment to generate the final standardized labeling result, specifically including:

[0135] Step S51: input the degradation sample into the lightweight target detection to automatically identify the structure regions of the text area, caption area, annotation area, illustration area and seal area, and generate corresponding detection boxes and label information; then input the degradation sample into the OCR recognition network to obtain the labeled text content, language attribute and degradation parameters corresponding to the image.

[0136] The labeling result of this step is automatically output in JSON or XML format, one-to-one corresponding to the image, and the fields include: {uuid, language, element_type, coordinates, degradation_params, text_content}.

[0137] Step S52: perform OCR recognition on the labeled text content again to obtain the read-back result; calculate the edit distance and semantic matching degree between the two to obtain the read-back consistency score, and when the read-back consistency score is lower than the preset threshold, trigger local re-recognition or re-labeling operation.

[0138] This step first performs OCR recognition on the text area that has been automatically labeled to verify the accuracy and consistency of the preliminary labeling result. The text area that has been labeled refers to all text areas identified and marked in the initial labeling process of step S51, including text content and their position coordinates in the page (e.g., the position of text boxes, paragraph order, etc.). In the read-back phase of this step, OCR recognition is performed on these labeled text areas, i.e., the text in these areas is identified again using OCR technology, and the obtained text is the read-back result. The read-back result is the text content obtained after the second OCR identification, which is compared with the initially labeled text content (i.e., the original labeled text). The present application calculates the difference between the original labeled text and the read-back result in the following ways:

[0139] 1. Levenshtein Distance: This method calculates the character-level differences between the original annotated text and the read-back result, measuring their similarity. The smaller the edit distance, the closer the read-back result is to the original annotated text.

[0140] 2. Semantic Matching Degree: This method is based on text embedding technology and calculates the semantic similarity between the original annotated text and the read-back result. The higher the semantic matching degree, the more consistent the content and meaning of the two.

[0141] Based on the above two indicators, the read-back consistency score is obtained. When the read-back consistency score is lower than the preset threshold (e.g., 90%), a local re-identification or re-labeling operation is automatically triggered. This operation targets areas that may have annotation errors (such as text omissions, character recognition errors, etc.), re-executes OCR identification, and updates the annotation content to ensure that the annotation results are completely consistent at the character and semantic levels. Through the OCR read-back consistency detection step, semantic layer verification and primary correction are achieved, which can effectively improve the annotation accuracy and ensure the consistency of the original annotated text and the read-back result in terms of content, structure, and semantic level, providing reliable data support for subsequent steps.

[0142] Step S53: Calculate the Intersection-over-Union (IOU) between each pair of detection boxes. When the IOU exceeds the threshold, it is determined that the two are duplicate or overlapping annotations, and then automatically perform box merging, boundary smoothing, and coordinate adjustment operations.

[0143] After completing the text content verification, this step analyzes the spatial distribution of all detection boxes to identify overlapping and offset problems in the structure boundary. Calculate the Intersection-over-Union (IOU) between each pair of detection boxes. When the IOU exceeds the threshold of 0.85, it is automatically determined as duplicate or overlapping annotations. In this case, according to the detection confidence, box area, and hierarchical structure relationship (such as the main text area being prior to the comment area), automatically perform box merging, boundary smoothing, and coordinate adjustment operations. After correction, a new structure annotation index table is generated, making each region have a unique position and reasonable boundary on the page. This step ensures the accuracy of the annotation structure in spatial position and provides a geometric basis for subsequent model consistency determination.

[0144] Step S54: Call the main detection model and verification model in parallel for the same page image, respectively perform independent annotation tasks; compare the three key results of structure category, boundary coordinates, and text content output by the two models; when the difference exceeds the threshold, take the high-confidence result as the main output.

[0145] After the semantic and spatial two-layer verification of the above steps is completed, the application will call the main detection model and the verification model in parallel, perform independent labeling tasks on the same page image, and compare the three key results of structure category, boundary coordinates and text content output by the two models. When the difference exceeds 5%, the high confidence result is output as the main output, and the difference sample is recorded to the verification cache area. The cache data participates in model parameter update and adaptive learning regularly, which is used to gradually reduce the cross-model bias. This step is used as the final verification stage to dynamically compare and feedback the potential errors that cannot be completely corrected in the previous two steps, realize model-level self-correction, and further improve the labeling reliability of the system as a whole.

[0146] Through the progressive control of the above S52-S53 steps, a closed-loop verification process of "text consistency verification - spatial boundary correction - model difference comparison" is formed in the labeling stage, realizing the joint constraint of semantic, geometric and algorithmic layers.

[0147] Under this mechanism, the overall accuracy of the automatic labeling of the application remains above 95%, the structure IOU is stable above 0.85, and the OCR consistency rate is maintained above 90%. This mechanism ensures that the automatic labeling results have verifiability and reproducibility in the three dimensions of semantic matching, boundary positioning and structure recognition, providing reliable precision control support for multi-lingual ancient book high-fidelity data generation.

[0148] Finally, the application integrates the image, corpus, labeling and degradation parameters to form a unified data storage structure. The data set is divided by language, degradation type and task category, supporting multiple sub-tasks such as OCR recognition, layout analysis, repair prediction, etc. A data statistics report is automatically generated, including sample quantity, language proportion, degradation level distribution and labeling accuracy index. The final output data set contains about 300,000 training images and 50,000 verification images, covering three main ancient writing systems and multiple mixed layout styles, providing high-quality data support for cross-language OCR research.

[0149] In the application layer, the application has good implementability and industrialization foundation, and can be directly embedded in multiple production, education and cultural relic protection scenarios.

[0150] (1) Cultural relic protection and archive digitization scene: The application can be deployed in the digitization editing process of museums, libraries and archives, and run as a "high-fidelity data enhancement and automatic labeling module". The system can generate structured ancient book data in batches under the condition of few real samples by using reverse synthesis and automatic labeling mechanism, and provide reliable support for damaged page repair, defect identification and version comparison. Actual test shows that the system can reduce the manual labeling labor cost by about 70%-90%, shorten the data preparation period by 50%-70%, and at the same time, the model recognition accuracy is improved by about 12%-20% on real collection samples, significantly improving the production efficiency and economic benefit of ancient book digitization.

[0151] (2) Education and scientific research application scene: The system can provide standardized multilingual ancient book data set and evaluation baseline for university courses, scientific research projects and national language informatization training, support multilingual OCR algorithm teaching and model verification. Through the "synthesis labeling" mechanism, teachers and researchers can quickly generate experimental samples, reduce experimental preparation and manual labeling workload by about 60%, and promote the popularization and application of multilingual literature research and intelligent recognition technology.

[0152] (3) Special application examples:

[0153] The technical system of the application has been tested and verified in multiple actual scenes, and can realize stable operation under different languages, task types and data scales, including:

[0154] ① Tibetan Buddhist scripture damaged page repair training set generation:

[0155] In the Tibetan literature repair project, traditional training data acquisition depends on manual scanning and post-labeling, and the data scale is small and the degraded sample is scarce. The application generates a training sample set of multiple damage types and degradation intensities based on a small amount of real samples by reverse synthesis and degradation modeling (S3). Combined with automatic labeling output (S5), a ternary data structure of "clear page-degraded page-repair target" is formed, which provides high-consistency input samples for scroll-type literature repair algorithms.

[0156] Actual test shows that the generated samples achieve 0.93 and 0.87 in texture consistency (SSIM) and structural accuracy (Structural IoU), respectively, and the recognition accuracy of the repair model is improved by about 18% compared with the artificial data set, and the manual labeling time is reduced by about 75%.

[0157] ② Museum and archive digitization data enhancement:

[0158] The invention is embedded as a data augmentation module in the existing digitization workflow for multilingual ancient book scanning data of museum collections and local archives. The invention automatically generates "real style simulation pages" based on the texture characteristics of real samples and outputs structured annotation files (JSON / XML). The augmented data can be directly used to train OCR, layout analysis and format recognition models.

[0159] In the national-level archive digitization pilot, after applying the augmented data generated by the invention, the average recognition rate of the OCR model on low-definition samples increased by about 15%, and the manual verification cost decreased by about 80%, verifying the industrial adaptability and economic feasibility of the technology.

[0160] ③ Education and scientific research examples:

[0161] Colleges and universities can quickly build ancient book data sets for teaching based on the invention, support OCR courses, ethnic language informatization training and scientific research paper reproduction. The invention supports parameter visualization and degradation control, which can provide students with full-process experimental cases from "corpus generation-image rendering-degradation simulation-structure annotation", promoting the construction of multilingual AI teaching system. In teaching tests, the experimental preparation time is reduced by about 60%, and the accuracy of the student model in the multi-lingual OCR task is improved by about 10% on average.

[0162] In summary, the invention drives the logic path through reverse synthesis, which is different from the traditional forward rendering method in the generation direction, and realizes controllable mapping and closed-loop optimization from real distribution characteristics to data generation. This method innovatively combines document restoration and multi-lingual OCR generation ideas in the academic field, and realizes an automatic pipeline with adjustable parameters, measurable performance and expandable languages in the engineering field, providing efficient and sustainable technical support for the intelligent protection and inheritance of Chinese historical documents.

[0163] Embodiment two:

[0164] As shown in Figure 2 , the embodiment of the invention provides a reverse synthesis driven multilingual ancient book high-fidelity annotation generation system, including the following modules:

[0165] The data acquisition module 61 is used to collect multilingual ancient book text data from multiple sources; through language tokenization, noise cleaning and structure standardization processing, an OCR corpus covering multiple languages is constructed;

[0166] The rendering module 62 is used to perform typesetting modeling and visual rendering using a multilingual structure modeling algorithm based on the writing rules specific to each language, and to realize preliminary structure-level annotation according to the spatial hierarchical relationship of the main text, title, annotations, seals and illustrations;

[0167] The visual simulation and degradation module 63 is configured to construct a degradation model for the degraded sample, and to precisely adjust the degradation type, intensity and distribution by multi-parameter mapping and combination control, so that the degraded sample is highly consistent with the real ancient books in terms of texture, brightness and structure.

[0168] The quality evaluation and reverse feedback module 64 is configured to design a multi-dimensional quality evaluation and reverse feedback mechanism, to automatically evaluate the quality of the degraded sample, and to dynamically self-correct and close-loop optimize according to the evaluation result, so as to obtain the final degraded sample.

[0169] The labeling and structure identification module 65 is configured to use a lightweight target detection and OCR identification network to perform semantic-level labeling on the final degraded sample, and to generate a final standardized labeling result by combining with back-reading consistency detection, region IOU correction and multi-model consistency determination.

[0170] A reverse synthesis driven multilingual ancient book high-fidelity labeling generation device includes one or more electronic devices, wherein the one or more electronic devices are configured to implement a reverse synthesis driven multilingual ancient book high-fidelity labeling generation method.

[0171] An electronic device includes one or more processors, and a memory configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement a reverse synthesis driven multilingual ancient book high-fidelity labeling generation method.

[0172] A computer readable storage medium has stored thereon executable instructions that, when executed by a processor, cause the processor to implement a reverse synthesis driven multilingual ancient book high-fidelity labeling generation method.

[0173] A non-transitory computer readable storage medium has stored thereon a computer program that, when executed by a processor, implements a reverse synthesis driven multilingual ancient book high-fidelity labeling generation method.

[0174] The above is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application should not be limited to these embodiments shown herein, but should be consistent with the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis, characterized in that, include: Step S1: Collect multilingual ancient text data from multiple sources; By performing language segmentation, noise removal, and structural standardization, an OCR corpus covering multiple languages ​​is constructed. Step S2: Based on the unique writing rules of each language, use multilingual structural modeling algorithms to perform typesetting modeling and visual rendering, and achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations; Step S3: Construct a degradation model for the degradation sample. Through multi-parameter mapping and combination control, achieve precise adjustment of degradation type, intensity and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness and structure. Step S4: Design a multi-dimensional quality assessment and reverse feedback mechanism to automatically assess the quality of the degraded samples, and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples; Step S5: Use a lightweight object detection and OCR recognition network to perform semantic-level annotation on the final degraded sample, and combine readback consistency detection, regional IOU correction and multi-model consistency judgment to generate the final standardized annotation result.

2. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 1, characterized in that, Step S2: Based on the unique writing rules of each language, a multilingual structural modeling algorithm is used for typesetting modeling and visual rendering. Preliminary structural annotations are achieved based on the spatial hierarchy of the main text, title tags, annotations, seals, and illustrations. Specifically, this includes: Step S21: Design a basic template to provide a general framework for multilingual typesetting; Step S22: Construct a unified intermediate layout representation layer (ILR) to abstractly describe the spatial structure and writing attributes of ancient book pages in different languages; Step S23: Based on the differences in the writing systems of Mongolian, Tibetan and Manchu languages, define the corresponding structural mapping rules and parameterization control, and map the ILR parameters to the language-specific template parameter set; Step S24: Design a dynamic expansion mechanism for template parameters to support the rapid access and automatic adaptation of new languages. When a new language is identified, it is only necessary to supplement the differences between it and the basic template, and the template parameter set for that language will be automatically generated through an adaptive function.

3. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 2, characterized in that, Step S3: Constructing a degradation model for the degradation sample, and through multi-parameter mapping and combination control, achieving precise adjustment of degradation type, intensity, and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness, and structure, specifically including: Step S31: Define a degradation parameter triplet (α, β, γ) to correspond to three degradation factors: aging attenuation, illumination disturbance, and local damage, respectively. α, β, and γ are non-negative real numbers. Normalize the parameter triplet to obtain: α′ = α / (α+β+γ), β′ = β / (α+β+γ), γ′ = γ / (α+β+γ), where α+β+γ > 0. Step S32: Construct the degradation mapping function f(·) to generate the final degradation sample: I deg =f(I syn ,a,b,c)=I syn ×(1 - δ(α′A + β′L))× (1 - γ′N) + γ′B × N; Among them, I syn For input, a clear synthetic image; I deg The output is the degraded image; A, L, and N are the aging mapping parameters, illumination mapping parameters, and noise mask parameters extracted from the statistics of real ancient book samples, respectively; B is the background fill value, used to simulate the realistic appearance of the hole area; δ is the attenuation adjustment factor.

4. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 3, characterized in that, Step S4: Design a multi-dimensional quality assessment and reverse feedback mechanism to automatically evaluate the quality of the degraded samples, and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples, specifically including: Step S41: Image fidelity metrics: Structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) are used to measure the similarity between the generated degraded samples and the real ancient books in terms of texture, lighting, and edges. Step S42: Semantic consistency index: Introduce the cross-modal semantic matching index CLIPScore to evaluate the matching degree between the visual features of the generated samples and the semantic space of the text; Step S43: Structural annotation accuracy indicators: The accuracy of automatically annotated regions and structures is evaluated by using OCR recognition accuracy and structural intersection-union ratio (IoU). Step S44: Define the overall quality scoring function: Q=w1×SSIM+w2×PSNR+w3×CLIPScore+w4×OCRAcc+w5×SioU; Where w1~w5 are empirical weights, satisfying ∑w i =1, i=1-5; OCRACcc is the OCR recognition accuracy; SioU is the intersection-union ratio; Set minimum threshold Q min When the degraded sample is below the threshold, the process proceeds to step S3 for parameter adjustment. Specifically, this involves triggering an automatic parameter adjustment process, including: performing a constrained parameter search or gradient approximation update based on Q for (α,β,γ), updating the mapping weights and threshold set of the degradation control function, and regenerating the degraded sample until Q≥Q. min .

5. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 4, characterized in that, Step S5: Using a lightweight object detection and OCR recognition network, semantic-level annotation is performed on the final degraded samples. Combined with readback consistency detection, region IOU correction, and multi-model consistency determination, the final standardized annotation result is generated, specifically including: Step S51: Input the degraded sample into a lightweight object detection system to automatically identify the structural regions of the main text area, title area, annotation area, illustration area, and stamp area, and generate corresponding detection boxes and label information; then input the degraded sample into an OCR recognition network to obtain the labeled text content, language attributes, and degradation parameters corresponding to the image; Step S52: Perform OCR recognition on the annotated text content again to obtain the reread result; calculate the edit distance and semantic matching degree between the two to obtain the reread consistency score. When the reread consistency score is lower than the preset threshold, trigger the local re-recognition or re-annotation operation. Step S53: Calculate the Intersection over Union (IOU) between each pair of detection boxes. When the IOU exceeds the threshold, it is determined that the two boxes are duplicate or overlapping, and then the boxes are automatically merged, the boundaries are smoothed, and the coordinates are adjusted. Step S54: Call the main detection model and the validation model in parallel for the same page image and perform independent annotation tasks respectively; compare the three key results output by the two models: structure category, boundary coordinates and text content; when the difference exceeds the threshold, take the high confidence result as the main output.

6. A reverse synthesis-driven multilingual ancient text high-fidelity annotation generation system, characterized in that, Includes the following modules: The data acquisition module is used to collect multilingual ancient text data from multiple sources; through language segmentation, noise removal and structural standardization, an OCR corpus covering multiple languages ​​is constructed. The rendering module is used to perform typesetting modeling and visual rendering based on the unique writing rules of each language, using multilingual structural modeling algorithms, and to achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations. The visual simulation and degradation module is used to construct degradation models for degradation samples. Through multi-parameter mapping and combination control, it can accurately adjust the degradation type, intensity and distribution, so that the degradation samples are highly consistent with real ancient books in terms of texture, brightness and structure. The quality assessment and feedback module is used to design a multi-dimensional quality assessment and feedback mechanism to automatically assess the quality of the degraded samples and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples. The annotation and structure recognition module is used to perform semantic-level annotation on the final degraded sample using a lightweight object detection and OCR recognition network, and to generate the final standardized annotation result by combining readback consistency detection, region IOU correction and multi-model consistency judgment.

7. A reverse synthesis-driven multilingual ancient book high-fidelity annotation generation device, characterized in that, It includes one or more electronic devices, wherein the one or more electronic devices are used to implement the method of any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5.

Images