Reverse synthesis driven multi-language ancient book high-fidelity label generation method and system

Abstract

The invention provides a reverse synthesis driven multi-language ancient book high-fidelity label generation method and system, and belongs to the technical field of document image processing, and the method comprises the steps: S1, constructing an OCR corpus covering multiple languages; s2, performing typesetting modeling and visual rendering by using a multilingual structure modeling algorithm based on specific writing rules of all languages; s3, constructing a degradation model for a degradation sample, and realizing accurate adjustment of degradation type, intensity and distribution through multi-parameter mapping and combined control; s4, performing automatic quality evaluation on the degraded sample, and performing dynamic self-correction and closed-loop optimization according to an evaluation result to obtain a final degraded sample; and S5, performing semantic-level labeling on the final degraded sample by using a lightweight target detection and OCR recognition network to generate a final standardized labeling result. The method disclosed by the invention provides a solid technical basis for constructing a high-quality and multi-language historical literature database.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of document image processing, and particularly relates to a multi-lingual ancient book high-fidelity labeling generation method and system driven by reverse synthesis. BACKGROUND

[0002] Early digitalization of ancient books mainly adopts heuristic rule methods such as page template matching, edge detection and region segmentation, and the text area and image area are segmented by manually setting page structure rules. Such methods perform well in standardized printed documents, but when faced with irregular layout, misaligned text, overlapping annotations, paper deformation and other situations in ancient books, the rules are easily invalid, and it is difficult to realize large-scale automatic processing. Representative works include the Chinese Reborn Ancient Book Digitalization Project of the National Library of China and some page structure analysis methods based on template matching.

[0003] With the development of computer vision, researchers began to use convolutional neural networks (CNN), fully convolutional networks (FCN) and object detection models (such as U-Net, YOLO, Transformer OCR) for document recognition and repair tasks. Typical methods include image enhancement, denoising, binarization, distortion correction and shadow removal. Such methods can improve the readability of ancient books at the visual level, but are limited by local receptive fields and are difficult to capture long-range layout relationships and character structures in complex backgrounds. For example, the DiffHDR model proposed by Yang et al. uses a diffusion network to achieve image-level repair of damaged ancient books, and improves repair consistency by introducing semantic and spatial conditions; but its training relies on a large number of manually annotated samples and is mainly aimed at single-lingual historical documents.

[0004] To overcome the limitations of single-modal repair, researchers propose a multi-stage scheme combining OCR, language models and generative networks. The AutoHDR framework proposes a three-stage fusion process: ① OCR-assisted damage localization (OADL); ② language model-driven content prediction (DCP); ③ image generative appearance reconstruction (HAR). This method significantly improves the automation of repair, and in the case of severe damage, the recognition rate is improved from 46.83% to 84.05%. However, its training still relies on manually collected and true repair samples, with high data acquisition costs, and limited generalization ability in multi-lingual environments.

[0005] In recent years, researchers have begun to explore synthetic data-driven OCR training paradigms. For example, Deep Aramaic (ICDAR 2025) proposed by Ding et al. generates high-fidelity inscription data through 3D glyph modeling and weathering simulation; SARD Dataset (arXiv:2505.24600) developed by Ahmed et al. generates large-scale Arabic OCR samples using procedural typesetting. Although these methods have made progress in monolingual documents, they have failed to simulate the physical degradation characteristics and multi-element layout structure of ancient books, and lack automated annotation mechanisms, limiting their application in complex ancient book scenarios.

[0006] However, the aforementioned prior art has the following drawbacks:

[0007] (1) Data scarcity and lack of realism: Existing ancient book recognition and restoration models rely on manual scanning and annotation of data. However, the available samples of ancient books in languages ​​such as Mongolian, Tibetan, and Manchu are extremely limited, making it difficult for the models to generalize to real-world scenarios. Although synthetic data methods can supplement the samples, they often lack physical characteristics such as paper texture, ink diffusion, and aging light and shadow, resulting in a significant difference in distribution between the generated samples and real ancient books.

[0008] (2) Insufficient language structure modeling: Most synthesis methods are geared towards modern printed text or monolingual documents with simple layout rules, which cannot handle the vertical continuous writing of Mongolian, the compound stacking style of Tibetan, and the horizontal symbol system of Manchu. The lack of multilingual script structure leads to distortion in the text form and layout of the generated data, making it difficult to meet the training requirements of multilingual OCR models.

[0009] (3) Lack of automated annotation and controllable degradation mechanism: Traditional synthetic samples usually only generate image layer information and do not provide structural annotations such as characters, lines, annotations, and seals; at the same time, the degradation process often relies on random blurring or noise addition, and it is impossible to accurately control the type and intensity of damage, making it difficult to use the data for structured training and evaluation.

[0010] (4) Fragmented processing flow and insufficient scalability: Existing document generation, repair and recognition technologies are mostly independent modules, lacking a unified automated pipeline. They cannot achieve continuous processing from content generation to structural annotation, nor do they have language adaptation capabilities and scalable data fusion mechanisms. Summary of the Invention

[0011] To address the aforementioned technical problems, this invention provides a reverse synthesis-driven method for generating high-fidelity annotations for multilingual ancient texts, comprising the following steps:

[0012] Step S1: Collect multilingual ancient text data from multiple sources; construct an OCR corpus covering multiple languages ​​through language segmentation, noise removal, and structural standardization.

[0013] Step S2: Based on the unique writing rules of each language, use multilingual structural modeling algorithms to perform typesetting modeling and visual rendering, and achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations;

[0014] Step S3: Construct a degradation model for the degradation sample. Through multi-parameter mapping and combination control, achieve precise adjustment of degradation type, intensity and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness and structure.

[0015] Step S4: Design a multi-dimensional quality assessment and reverse feedback mechanism to automatically assess the quality of the degraded samples, and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples;

[0016] Step S5: Use a lightweight object detection and OCR recognition network to perform semantic-level annotation on the final degraded sample, and combine readback consistency detection, regional IOU correction and multi-model consistency judgment to generate the final standardized annotation result.

[0017] Beneficial effects:

[0018] 1. This invention provides a reverse synthesis-driven method for generating high-fidelity annotations for multilingual ancient books. Starting from the visual attributes of real ancient book samples, the generation process is controlled by parameter backtracking, forming a bottom-up, closed-loop reverse generation path.

[0019] 2. This invention addresses the differences in language systems such as Mongolian, Tibetan, and Manchu by establishing parameterized typesetting templates to achieve automatic language conversion.

[0020] 3. This invention simulates the degradation process of paper and ink through a physical parameterization model, generating images that are visually highly similar to real ancient books, and combines indicators such as PSNR, SSIM, CLIPScore, OCR Accuracy, and Structural IoU to complete quality feedback.

[0021] 4. This invention achieves simultaneous output of synthesis and annotation through target detection network and OCR recognition network, significantly reducing manual intervention.

[0022] 5. This invention supports multiple languages, mixed layouts, and multi-task output, and has cross-domain adaptability and data reuse capabilities.

[0023] 6. The reverse synthesis method of the present invention takes the visual and physical feature distribution of real ancient books as the starting point, and realizes the reverse modeling from real samples to synthetic data through parameter backtracking, forming a closed-loop control path of "real sample feature distribution → parameter backtracking → controllable generation → quality feedback → automatic annotation", providing unified data support for multilingual ancient book recognition, digital protection and intelligent utilization. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a reverse synthesis-driven method for generating high-fidelity annotations for multilingual ancient books according to the present invention.

[0025] Figure 2 This is a structural block diagram of a reverse synthesis-driven multilingual ancient book high-fidelity annotation generation system according to the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0027] Example 1:

[0028] like Figure 1 As shown in the figure, the reverse synthesis-driven method for generating high-fidelity annotations for multilingual ancient books provided by this invention includes the following steps:

[0029] Step S1: Collect multilingual ancient text data from multiple sources; construct an OCR corpus covering multiple languages ​​through language segmentation, noise removal, and structural standardization.

[0030] Step S2: Based on the unique writing rules of each language, use multilingual structural modeling algorithms to perform typesetting modeling and visual rendering, and achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations;

[0031] Step S3: Construct a degradation model for the degradation sample. Through multi-parameter mapping and combination control, achieve precise adjustment of degradation type, intensity and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness and structure.

[0032] Step S4: Design a multi-dimensional quality assessment and reverse feedback mechanism to automatically assess the quality of degraded samples, and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples;

[0033] Step S5: Use a lightweight object detection and OCR recognition network to perform semantic-level annotation on the final degraded samples, and combine readback consistency detection, region IOU correction and multi-model consistency judgment to generate the final standardized annotation results.

[0034] In one embodiment, step S1 above: collecting multilingual ancient text data from multiple sources; constructing an OCR corpus covering multiple languages ​​through language segmentation, noise removal, and structural standardization, specifically including:

[0035] This invention collects ancient text data in Mongolian, Tibetan, Manchu, and other languages ​​from multiple sources, including open corpora, OCR-extracted text, manually entered materials, and customized documents. Through language segmentation, noise removal, and structural standardization, a multilingual OCR corpus covering various languages ​​is constructed. This invention supports multiple methods such as automatic crawling, manual input, and file import, ensuring the diversity of corpus sources and the authenticity of content, providing a reliable semantic foundation for subsequent data generation. Unlike traditional methods, the corpus collected in this step not only serves text rendering but is also used for subsequent alignment with the feature distribution of real samples, achieving integrated semantic and visual generation preparation.

[0036] In one embodiment, step S2 above involves using a multilingual structural modeling algorithm to perform typesetting modeling and visual rendering based on the unique writing rules of each language, and to achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations. Specifically, this includes:

[0037] Step S21: Design a basic template to provide a general framework for multilingual typesetting.

[0038] In the process of digitizing and typesetting ancient books in multiple languages, traditional typesetting methods struggle to uniformly address the typesetting needs of different languages ​​due to significant differences in writing direction, typesetting order, and character structure among languages ​​such as Mongolian, Tibetan, and Manchu. Therefore, to ensure the system can efficiently and consistently generate typesettings for multilingual ancient books, this invention designs a basic template that provides a universal framework and flexible extensibility for multilingual typesetting.

[0039] The basic template is a standardized set of parameters designed to accommodate multilingual typesetting needs. Its main goal is to provide a unified typesetting framework to ensure the accuracy, scalability, and consistency of multilingual ancient book typesetting. This template allows for maintaining visual consistency while flexibly addressing the specific typesetting requirements of each language when handling multiple languages.

[0040] (1) Core components of the basic template:

[0041] 1. Layout rules: including basic layout specifications such as writing direction (vertical or horizontal), line priority or column priority.

[0042] 2. Visual parameters: such as font size, character spacing, line spacing, etc., to ensure the readability and consistency of the text.

[0043] 3. Structure Definition: Define the structural elements on the page (such as body text, title tags, annotations, etc.) to ensure the rationality of the layout structure.

[0044] (2) The role of the basic template:

[0045] 1. Unified Framework: Provides a universal parameter setting for multilingual typesetting, ensuring consistent typesetting effects across different languages.

[0046] 2. Automatic Adaptability: The system can automatically adapt to new languages ​​based on the basic template, adjusting differences (such as writing direction, character density, etc.) to generate a typesetting template that meets the requirements of the new language.

[0047] 3. Simplified design: Avoid designing layout rules from scratch. By adjusting the differences from the basic template, quickly generate layout formats for new languages.

[0048] 4. Support for subsequent modeling: The basic template provides the necessary layout framework for structural modeling and automatic annotation, ensuring that subsequent steps can be executed accurately.

[0049] Step S22: Construct a unified intermediate layout representation layer (ILR) to abstractly describe the spatial structure and writing attributes of ancient book pages in different languages.

[0050] To achieve unified typesetting and structural mapping of ancient books in multiple languages ​​such as Mongolian, Tibetan, and Manchu, this invention designs a multilingual structural modeling algorithm in the rendering engine. This algorithm, centered on the "Intermediate Layout Representation (ILR)," achieves calculable expression and extensible generation of language differences through unified abstraction and parametric modeling of typesetting rules across scripts.

[0051] This invention first constructs a unified intermediate layout representation layer (ILR) to abstractly describe the spatial structure and writing attributes of ancient book pages in different languages. The ILR is a semantic-geometric intermediate layer shared across languages, and its data structure is shown in Table 1.

[0052] Table 1 ILR Data Structure Definition Table

[0053] Table 1 defines the core fields and parameter ranges of the Intermediate Page Layout (ILR). The ILR unifies the layout elements of pages in different languages ​​in a structured manner, and the logical relationships between its fields are as follows:

[0054] 1. The script_type determines the structure mapping rules for the call;

[0055] 2. Writing_direction and baseline_orientation together control the direction of text and layout;

[0056] 3. line_spacing, char_spacing, and char_density together reflect the writing density;

[0057] 4. The `anchor_graph` establishes the spatial hierarchy between various functional areas of the page and is a key interface for the subsequent automatic annotation stage. `anchor_graph` describes the hierarchy and adjacency relationships of areas such as text, title tags, annotations, illustrations, and seals in the form of nodes (V) and edges (E), serving as an important structural bridge connecting the subsequent automatic annotation module. The ILR, as a unified typesetting interface, inherits from the statistical characteristics of authentic ancient books and drives layout generation and annotation output, achieving language-independent structural expression.

[0058] Step S23: Based on the differences in the writing systems of Mongolian, Tibetan and Manchu languages, define corresponding structural mapping rules and parameterization controls, and map ILR parameters to language-specific template parameter sets.

[0059] To accommodate the differences in the writing systems of the three main languages, Mongolian, Tibetan, and Manchu, this invention defines corresponding structural mapping rules and parameterized control tables, as shown in Table 2.

[0060] Table 2 Structure Mapping Rules and Parameters

[0061] During the generation process, the corresponding rule set is automatically invoked based on script_type, mapping ILR parameters to language-specific template parameter sets. s This forms a unified structure to the rendering control path. The mapping function is defined as follows:

[0062] Φ s : ILR→ Θ s , Θ s = { θ grid , θ spacing , θ glyph , θ punct , θ anchor};

[0063] Where: θ grid For column grid and paragraph block layout; θ spacing Controls line spacing, character spacing, and column spacing; θglyph For variations in the aspect ratio and font size; θ punct Punctuation rotation and alignment strategy; θ anchor For structural block anchor points and hierarchies.

[0064] Step S24: Design a dynamic extension mechanism for template parameters to support the rapid access and automatic adaptation of new languages. When a new language is identified, it is only necessary to supplement the differences between it and the basic template, and the template parameter set for that language will be automatically generated through an adaptive function.

[0065] To enhance versatility, this invention proposes a dynamic template parameter expansion mechanism to support rapid access and automatic adaptation to new languages. When a new language s is identified... new At that time, only the difference item Δ between it and the basic template needs to be added. script (Including key parameters such as writing direction, baseline direction, and character density), which can be achieved through an adaptive function:

[0066] P new = f adapt (ILR) base , Δ script );

[0067] Among them, f adapt It is an adaptive function; ILR base ILR based on template; P new As a template for new languages;

[0068] Automatically generate the typesetting template parameter set for this language. new This mechanism enables the method to generalize to "language as template," supporting the generation of layouts for newly added or mixed-language scripts without manual adjustments. The extended logic is as follows:

[0069] 1. For vertical text, the system prioritizes column-first layout;

[0070] 2. If char_density is too high, then automatically reduce line_spacing;

[0071] 3. If the newly added script has annotation or chaining features, then enable the ligature_mode or diacritic_policy parameter group.

[0072] This mechanism significantly improves portability and versatility in tasks involving new languages, mixed scripts, or cross-disciplinary ancient book typesetting, ensuring consistency and automation in page generation.

[0073] The layout parameters in this step do not rely on manual presets, but are dynamically driven by the statistical distribution generated through parameter backtracking of real sample features. The rendering engine introduces font randomization and style perturbation mechanisms during the generation process, resulting in visually rich diversity and naturalness in the generated images. The rendered images not only contain the main text but can also overlay elements such as annotations, titles, illustrations, and seals, thus forming complex and realistic ancient book page layouts.

[0074] In one embodiment, step S3 above: constructing a degradation model for the degradation sample, and through multi-parameter mapping and combination control, achieving precise adjustment of degradation type, intensity, and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness, and structure, specifically including:

[0075] Step S31: Define a degradation parameter triplet (α, β, γ) to correspond to three degradation factors: aging attenuation, illumination disturbance, and local damage, respectively. α, β, and γ are non-negative real numbers. Normalize the parameter triplet to obtain: α′ = α / (α+β+γ), β′ = β / (α+β+γ), γ′ = γ / (α+β+γ), where α+β+γ > 0.

[0076] The parameter range for the triplet is set as follows:

[0077] 1. Aging parameter α ∈ [0.2, 0.8]: controls the time-related decay of paper, such as yellowing, fiber coarsening, and ink penetration;

[0078] 2. Illumination parameter β ∈ [0.1, 0.6]: Adjusts the distribution of light and shadow, the direction of shadows, and the brightness deviation;

[0079] 3. Damage parameter γ ∈ [0, 0.3]: simulates local structural damage such as insect holes, mold spots and stains.

[0080] Step S32: Construct the degradation mapping function f(·) to generate the final degradation sample:

[0081] Ideg=f(Isyn,α,β,γ)=Isyn×(1 - δ(α′A + β′L)) × (1 - γ′N) + γ′B ×N;

[0082] Among them, I syn For input, a clear synthetic image; I degThe output is the degraded image; A, L, and N are the aging mapping parameters, illumination mapping parameters, and noise mask parameters extracted from real ancient book samples, respectively; B is the background fill value, used to simulate the realistic representation of the hole area; to avoid image value inversion due to coefficient amplification after normalization, this invention introduces an attenuation adjustment factor δ ∈ (0, 1], used to control the total amplitude of degradation intensity and ensure that the multiplicative term is always non-negative in a physical sense; in this embodiment of the invention, the value of δ is between [0.5, 0.9]. The above mapping process can also be regarded as a specific implementation of the degradation control function f(·), whose core goal is to integrate the three types of degradation factors—aging, illumination, and damage—into a unified visual generation path.

[0083] In the above formula, A, L, N, and B are key parameters in the degradation process, representing different types of degradation characteristics. Specifically:

[0084] 1. AgeMap (A aging mapping parameter) is a two-dimensional matrix extracted by analyzing real ancient book samples. Each element in the matrix represents the aging degree of a certain pixel in the image. By modeling physical degradation phenomena such as paper aging and ink diffusion, each value in the matrix represents the aging intensity of that area. The higher the value, the more severe the aging of that area, manifested as yellowing of paper, ink penetration, etc.

[0085] 2. LightMap (L: Light Mapping Parameters) is another two-dimensional matrix representing the light intensity or lighting perturbation of each pixel in the image. By extracting lighting features from real ancient book samples, LightMap simulates phenomena such as brightness changes and shadow effects caused by uneven lighting during the preservation process. Each element value in the matrix reflects the light intensity of that area in the image; a higher value indicates stronger lighting in that area, while a lower value indicates weaker lighting or the presence of shadows.

[0086] 3. NoiseMask (N noise mask parameters) is a binary matrix that represents whether a region in an image is affected by noise or damage (such as insect damage, mold, etc.). Each element in the matrix has a value of 0 or 1, where 1 indicates that the region has noise or damage, and 0 indicates that the region has no damage. Using NoiseMask, it is possible to simulate the local damage or noise commonly found in real ancient books in images, making the degradation effect more realistic.

[0087] 4. The background layer (B) is a background layer matrix with dimensions consistent with the input image Isyn (usually H×W×C, where C is the number of channels). It is used for visual filling or simulation replacement within the damaged areas marked by the degradation mask N, thereby simulating effects such as missing paper, holes, or structural damage. The element values ​​of B range from [0,1], representing the normalized intensity values ​​of image pixels. For grayscale filling scenarios, B can be a constant matrix (e.g., B=0.85); for texture filling, B can be generated from paper texture blocks extracted from real ancient book images and normalized to ensure consistency with the degradation image. Its generation includes two paths:

[0088] (1) The paper texture extracted from real ancient book images is used to construct a background texture library. During the degradation process, the system samples texture blocks of corresponding size from the damaged area according to the location of the damaged area. After scaling and edge feathering, the textures are used for area filling.

[0089] (2) For small damaged areas, the background content of the missing area can be dynamically generated by edge pixel interpolation or image repair algorithm (such as the inpainting method based on Telea or Navier-Stokes) to improve the local texture continuity and overall visual naturalness.

[0090] These four matrices (A, L, N, B) enable precise control over the degradation process, simulating complex degradation phenomena such as paper aging, uneven lighting, and localized damage. During image generation, the system integrates the effects of these three degradation factors to generate a degraded image, ensuring that the generated ancient book images are highly close to reality both visually and physically.

[0091] This invention extracts aging texture distribution, illumination attenuation patterns, and damage noise features from authentic ancient books, constructs a multidimensional mapping matrix, and achieves continuous regulation of the degradation process through the following degradation control function: Input: Clean image I_syn, degradation parameters (α, β, γ) A←Extract(AgeMap from texture database) L←Sample(LightMap from illumination set) N←Generate(NoiseMask based on damage statistics) B←Select(background fill texture) I_deg←I_syn × (1 - δ(α′A + β′L)) × (1 - γ′N) + γ′B × N Output: Degraded image I_deg

[0092] This step aims to perform controlled degradation simulation on rendered ancient book page samples to mimic the natural degradation phenomena that occur in real ancient books during long-term preservation due to factors such as paper aging, ink diffusion, uneven lighting, insect damage, and mold stains. The core of this step is to establish a parametric degradation model. Through multi-parameter mapping and combination control, precise adjustment of the degradation type, intensity, and distribution can be achieved, ensuring that the generated samples are highly consistent with real ancient books in terms of texture, brightness, and structure.

[0093] This invention achieves parameterization, controllability, and quantification of degradation simulation, overcoming the uncontrollability and unverifiability of traditional random degradation methods, and providing high-quality "clean sample - degraded sample - repair target" triplet data support for subsequent automatic annotation and OCR training.

[0094] To ensure that the generated samples achieve quantifiable high fidelity standards at the visual, structural, and semantic levels, this invention designs a multi-dimensional quality evaluation and reverse feedback mechanism with the following steps. After completing the degradation simulation, an automated quality assessment is performed on each batch of samples, and the assessment results are fed back to the front-end feature modeling and degradation parameter control layer in real time, realizing dynamic self-correction and closed-loop optimization of data generation.

[0095] In one embodiment, step S4 above—designing a multi-dimensional quality assessment and reverse feedback mechanism to automatically assess the quality of degraded samples, and performing dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples—specifically includes:

[0096] Step S41: Image fidelity metrics: Structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used to measure the similarity between the generated degraded samples and the real ancient books in terms of texture, lighting, and edges.

[0097] Specifically, in order to objectively quantify the visual consistency between the generated degraded ancient book samples and the real ancient book samples, this invention considers two dimensions: pixel error statistics and human visual perception. (1) Peak signal-to-noise ratio (PSNR) PSNR is calculated based on mean squared error (MSE) and is primarily used to evaluate the performance of generated samples in terms of illumination consistency and background noise. A higher PSNR value means that the generated degraded sample closely matches the pixel intensity distribution of the real sample. Its calculation formula is as follows: ; ; in, and These represent the generated degraded sample and the real ancient book reference image, respectively, where i and j represent pixels in the image. , The height and width of the image. This is the maximum possible value for the image pixels (usually 255).

[0098] (2) Structural similarity (SSIM) SSIM is a perception-based metric that differs from simple pixel comparison, focusing on the degree to which text edge sharpness and paper texture structure are preserved. The closer the SSIM value is to 1, the more visually realistic the generated degraded sample is. Its calculation formula is as follows: ; in, To generate an image window, A real image window; , They are respectively , The average value (reflecting brightness); , Variance (reflecting contrast / texture);

[0099] Covariance (reflecting structural similarity); , To maintain a stable constant.

[0100] Step S42: Semantic consistency index: Introduce the cross-modal semantic matching index CLIPScore to evaluate the matching degree between the visual features of the generated samples and the semantic space of the text; The principle of semantic consistency calculation is as follows: A pre-trained cross-modal representation model is used to map images and text to a unified semantic space, and their respective feature vectors are extracted. The CLIPScore score is obtained by calculating the similarity between the feature vectors; a higher score indicates a higher semantic consistency between the image content and the text. The text description includes page-rendered text, recognized text, or a combination thereof. Scores can be calculated at the page level or region level, and the calculated scores are summarized to obtain the final index.

[0101] Step S43: Structural annotation accuracy indicators: The accuracy of automatically annotated regions and structures is evaluated by OCR recognition accuracy and structural intersection-union ratio (IoU).

[0102] Step S44: Define the overall quality scoring function:

[0103] Q=w1×SSIM+w2×PSNR+w3×CLIPScore+w4×OCRAcc+w5×SioU;

[0104] Where w1~w5 are empirical weights, satisfying ∑w i =1, i=1-5; OCRACc is the OCR recognition accuracy; SioU is the structural intersection-union ratio.

[0105] Set minimum threshold Q min When the number of degraded samples falls below the threshold, the process proceeds to step S3 for parameter adjustment. Specifically, this involves triggering an automatic parameter adjustment process, including: performing a constrained parameter search or gradient approximation update based on Q for (α,β,γ), updating the mapping weights and threshold set of the degradation control function, and regenerating degraded samples until Q ≥ Q. min The feedback logic is as follows: Input: Sample set S, evaluation metrics Q If Q < Q_min: Adjust (α, β, γ) via gradient feedback Update feature model F' Regenerate I_deg' Re-evaluate Q Else: Save sample and export metadata

[0106] This invention achieves continuous optimization of the degradation model through gradient feedback and parameter correction, forming a closed-loop path of "feature modeling - degradation generation - quality evaluation - parameter backtracking - regeneration", ultimately ensuring that the degradation sample maintains stable and consistent high-fidelity features in all dimensions.

[0107] In one embodiment, step S5 above: using a lightweight object detection and OCR recognition network to perform semantic-level annotation on the final degraded samples, and combining readback consistency detection, region IOU correction, and multi-model consistency determination to generate the final standardized annotation result, specifically including:

[0108] Step S51: Input the degraded sample into the lightweight object detection, automatically identify the structural regions of the text area, title area, annotation area, illustration area and seal area, and generate corresponding detection boxes and label information; then input the degraded sample into the OCR recognition network to obtain the labeled text content, language attributes and degradation parameters corresponding to the image.

[0109] The annotation results of this step are automatically output in JSON or XML format, corresponding one-to-one with the images. The fields include: {uuid, language, element_type, coordinates, degradation_params, text_content}.

[0110] Step S52: Perform OCR recognition on the annotated text content again to obtain the rereading result; calculate the rereading consistency score by calculating the edit distance and semantic matching degree between the two; when the rereading consistency score is lower than the preset threshold, trigger the local re-recognition or re-annotation operation.

[0111] This step first performs OCR recognition again on the automatically annotated text areas to verify the accuracy and consistency of the initial annotation results. The annotated text areas refer to all text areas identified and marked during the initial annotation process in step S51. These areas include the text content and their position coordinates on the page (e.g., text box positions, paragraph order, etc.). In the readback phase of this step, these annotated text areas undergo readback OCR recognition, that is, the text in these areas is recognized again using OCR technology. The resulting text is the readback result. The readback result is the text content obtained after the second OCR recognition, which is compared with the initially annotated text content (i.e., the original annotated text). This invention calculates the difference between the original annotated text and the readback result in the following way:

[0112] 1. Edit Distance (Levenshtein Distance): This method calculates the character-level difference between the original annotated text and the readback result, measuring their similarity. The smaller the edit distance, the closer the readback result is to the original annotated text.

[0113] 2. Semantic Matching Degree: This method is based on text embedding technology to calculate the semantic similarity between the original annotated text and the rereading result. The higher the semantic matching degree, the more consistent the two are in content and meaning.

[0114] Based on the above two indicators, a readback consistency score is calculated. When the readback consistency score is lower than a preset threshold (e.g., 90%), a local re-identification or re-annotation operation is automatically triggered. This operation re-executes OCR recognition and updates the annotation content for areas that may have annotation errors (such as text omissions, character recognition errors, etc.), ensuring that the annotation results are completely consistent at the character and semantic levels. Through this OCR readback consistency detection step, semantic layer verification and initial correction are achieved, which can effectively improve annotation accuracy and ensure the consistency of the original annotated text and the readback results at the content, structure, and semantic levels, providing reliable data support for subsequent steps.

[0115] Step S53: Calculate the Intersection over Union (IOU) between each pair of detection boxes. When the IOU exceeds the threshold, it is determined that the two boxes are duplicate or overlapping labels, and then the boxes are automatically merged, the boundaries are smoothed, and the coordinates are adjusted.

[0116] After text content validation, this step analyzes the spatial distribution of all bounding boxes to identify overlapping and offset structural boundaries. The Intersection-over-Union (IOU) ratio between each pair of bounding boxes is calculated. When the IOU exceeds a threshold of 0.85, it is automatically identified as a duplicate or overlapping annotation. In this case, based on the detection confidence level, box area, and hierarchical relationship (e.g., text areas take precedence over annotation areas), box merging, boundary smoothing, and coordinate adjustment operations are automatically performed. After correction, a structural annotation index table is regenerated, ensuring that each region has a unique location and reasonable boundaries on the page. This step ensures the accuracy of the spatial location of the annotated structures, providing a geometric basis for subsequent model consistency determination.

[0117] Step S54: Call the main detection model and the validation model in parallel for the same page image and perform independent annotation tasks respectively; compare the three key results output by the two models: structure category, boundary coordinates and text content; when the difference exceeds the threshold, take the high confidence result as the main output.

[0118] After the semantic and spatial verification steps described above, this invention calls the main detection model and the verification model in parallel to perform independent annotation tasks on the same page image, and compares the three key results output by the two models: structural category, boundary coordinates, and text content. When the difference exceeds a threshold of 5%, the high-confidence result is given as the main output, and the difference samples are recorded in the verification cache. The cached data is periodically used to participate in model parameter updates and adaptive learning to gradually reduce cross-model bias. This step, as the final verification stage, dynamically compares and provides feedback on potential errors that were not fully corrected in the first two steps, achieving model-level self-correction and further improving the overall annotation reliability of the system.

[0119] Through the progressive control of steps S52-S53 above, a closed-loop verification process of "text consistency verification - spatial boundary correction - model difference comparison" is formed in the annotation stage, realizing the joint constraints of semantic, geometric and algorithmic layers.

[0120] Under this mechanism, the overall accuracy of automatic annotation in this invention remains above 95%, the structural IOU remains stable above 0.85, and the OCR consistency rate remains above 90%. This mechanism ensures that the automatic annotation results are verifiable and reproducible in three dimensions: semantic matching, boundary localization, and structural recognition, providing reliable accuracy control support for the generation of high-fidelity data of multilingual ancient books.

[0121] Finally, this invention integrates images, corpora, annotations, and degradation parameters into a unified data storage structure. The dataset is categorized by language, degradation type, and task type, supporting various sub-tasks such as OCR recognition, layout analysis, and restoration prediction. A data statistics report is automatically generated, including sample size, language ratio, degradation level distribution, and annotation accuracy indicators. The final output dataset contains approximately 300,000 training images and 50,000 validation images, covering three major ancient writing systems and various mixed layouts, providing high-quality data support for cross-linguistic OCR research.

[0122] At the application level, this invention has good feasibility and industrialization foundation, and can be directly embedded in various production, education and cultural relic protection scenarios.

[0123] (1) Cultural Relics Protection and Archives Digitization Scenarios: This invention can be deployed in the digitization acquisition and editing processes of museums, libraries, and archives as a "high-fidelity data enhancement and automatic annotation module." Utilizing reverse synthesis and automatic annotation mechanisms, the system can generate structured ancient book data in batches with minimal real-world samples, providing reliable support for damaged page restoration, incompleteness identification, and version comparison. Actual testing shows that this system can reduce manual annotation costs by approximately 70%–90%, shorten data preparation time by 50%–70%, and improve model recognition accuracy by approximately 12%–20% on real-world collection samples, significantly improving the efficiency and economic benefits of ancient book digitization.

[0124] (2) Educational and scientific research application scenarios: The system can provide standardized multilingual ancient book datasets and evaluation baselines for university courses, scientific research projects and ethnic language information training, and support multilingual OCR algorithm teaching and model verification. Through the "synthesis and annotation" mechanism, teachers and researchers can quickly generate experimental samples, reducing the workload of experimental preparation and manual annotation by about 60%, and promoting the application of multilingual document research and intelligent recognition technology.

[0125] (3) Specific application examples:

[0126] The technical system of this invention has been tested and verified in multiple real-world scenarios, and can achieve stable operation under different languages, task types, and data scales, specifically including:

[0127] ① Generation of a training set for repairing damaged pages of Tibetan Buddhist scriptures:

[0128] In Tibetan document restoration projects, traditional training data acquisition relies on manual scanning and subsequent annotation, resulting in small data sets and a scarcity of degraded samples. This invention, through reverse synthesis and degradation modeling (S3), automatically generates a training sample set with multiple levels of damage types and degradation intensities based on a small number of real samples. Combined with automatic annotation output (S5), a triplet data structure of "clear page - degraded page - restoration target" is formed, providing highly consistent input samples for scroll-style document restoration algorithms.

[0129] Actual tests show that the generated samples achieve 0.93 and 0.87 in texture consistency (SSIM) and structural accuracy (Structural IoU), respectively. The recognition accuracy of the repair model is improved by about 18% compared with the manual dataset, and the manual annotation time is reduced by about 75%.

[0130] ② Enhanced digital data in museums and archives:

[0131] For multilingual ancient book scans from museum collections and local archives, this invention is embedded as a data augmentation module into existing digitization workflows. Based on the texture features of real samples, this invention automatically generates "realistic style simulation pages" and outputs structured annotation files (JSON / XML). The augmented data can be directly used to train OCR, layout analysis, and page layout recognition models.

[0132] In a national-level pilot project for digitizing archives, the enhanced data generated by this invention improved the average recognition rate of the OCR model on low-resolution samples by about 15% and reduced the cost of manual verification by about 80%, thus verifying the industrial adaptability and economic feasibility of the technology.

[0133] ③ Examples of education and scientific research:

[0134] Universities can use this invention to quickly build ancient text datasets for teaching, supporting OCR courses, information technology training in ethnic minority languages, and the reproduction of research papers. This invention supports parameter visualization and degradation control, providing students with a complete experimental process from "corpus generation—image rendering—degradation simulation—structural annotation," promoting the construction of a multilingual AI teaching system. In teaching tests, experimental preparation time was reduced by approximately 60%, and the accuracy of student models in multilingual OCR tasks improved by an average of approximately 10%.

[0135] In summary, this invention, through a reverse synthesis-driven logic path, differs from traditional forward rendering methods in its generation direction, achieving controllable mapping and closed-loop optimization from real-world distribution features to data generation. At the academic level, this method innovatively integrates document restoration and multilingual OCR generation approaches; at the engineering level, it achieves an automated pipeline with adjustable parameters, measurable performance, and scalable language support, providing efficient and sustainable technical support for the intelligent protection and inheritance of Chinese historical documents.

[0136] Example 2:

[0137] like Figure 2 As shown, this embodiment of the invention provides a reverse synthesis-driven multilingual ancient book high-fidelity annotation generation system, including the following modules:

[0138] The data acquisition module 61 is used to collect multilingual ancient text data from multiple sources; and to build an OCR corpus covering multiple languages ​​through language segmentation, noise removal and structural standardization.

[0139] The rendering module 62 is used to perform typesetting modeling and visual rendering based on the writing rules specific to each language, using multilingual structural modeling algorithms, and to achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations.

[0140] The visual simulation and degradation module 63 is used to build degradation models for degradation samples. Through multi-parameter mapping and combination control, it can accurately adjust the degradation type, intensity and distribution, so that the degradation samples are highly consistent with real ancient books in terms of texture, brightness and structure.

[0141] The quality assessment and feedback module 64 is used to design a multi-dimensional quality assessment and feedback mechanism to automatically assess the quality of degraded samples and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples.

[0142] The annotation and structure recognition module 65 is used to perform semantic-level annotation on the final degraded samples using a lightweight object detection and OCR recognition network, and to generate the final standardized annotation results by combining readback consistency detection, region IOU correction and multi-model consistency judgment.

[0143] A reverse synthesis-driven multilingual high-fidelity annotation generation device for ancient books includes one or more electronic devices, wherein the one or more electronic devices are used to implement the reverse synthesis-driven multilingual high-fidelity annotation generation method for ancient books.

[0144] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors enable the one or more processors to implement a reverse synthesis-driven method for generating high-fidelity annotations of multilingual ancient texts.

[0145] A computer-readable storage medium storing executable instructions that, when executed by a processor, enable the processor to implement a reverse synthesis-driven method for generating high-fidelity annotations for multilingual ancient texts.

[0146] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for generating high-fidelity annotations for multilingual ancient texts driven by reverse synthesis.

[0147] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis, characterized in that, include: Step S1: Collect multilingual ancient text data from multiple sources; By performing language segmentation, noise removal, and structural standardization, an OCR corpus covering multiple languages ​​is constructed. Step S2: Based on the unique writing rules of each language, use multilingual structural modeling algorithms to perform typesetting modeling and visual rendering, and achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations; Step S3: Construct a degradation model for the degradation sample. Through multi-parameter mapping and combination control, achieve precise adjustment of degradation type, intensity and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness and structure. Step S4: Design a multi-dimensional quality assessment and reverse feedback mechanism to automatically assess the quality of the degraded samples, and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples; Step S5: Use a lightweight object detection and OCR recognition network to perform semantic-level annotation on the final degraded sample, and combine readback consistency detection, regional IOU correction and multi-model consistency judgment to generate the final standardized annotation result.

2. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 1, characterized in that, Step S2: Based on the unique writing rules of each language, a multilingual structural modeling algorithm is used for typesetting modeling and visual rendering. Preliminary structural annotations are achieved based on the spatial hierarchy of the main text, title tags, annotations, seals, and illustrations. Specifically, this includes: Step S21: Design a basic template to provide a general framework for multilingual typesetting; Step S22: Construct a unified intermediate layout representation layer (ILR) to abstractly describe the spatial structure and writing attributes of ancient book pages in different languages; Step S23: Based on the differences in the writing systems of Mongolian, Tibetan and Manchu languages, define the corresponding structural mapping rules and parameterization control, and map the ILR parameters to the language-specific template parameter set; Step S24: Design a dynamic expansion mechanism for template parameters to support the rapid access and automatic adaptation of new languages. When a new language is identified, it is only necessary to supplement the differences between it and the basic template, and the template parameter set for that language will be automatically generated through an adaptive function.

3. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 3, characterized in that, Step S3: Constructing a degradation model for the degradation sample, and through multi-parameter mapping and combination control, achieving precise adjustment of degradation type, intensity, and distribution, so that the degradation sample is highly consistent with the real ancient book in terms of texture, brightness, and structure, specifically including: Step S31: Define a degradation parameter triplet (α, β, γ) to correspond to three degradation factors: aging attenuation, illumination disturbance, and local damage, respectively. α, β, and γ are non-negative real numbers. Normalize the parameter triplet to obtain: α′ = α / (α+β+γ), β′ = β / (α+β+γ), γ′ = γ / (α+β+γ), where α+β+γ > 0. Step S32: Construct the degradation mapping function f(·) to generate the final degradation sample: I deg =f(I syn ,a,b,c)=I syn ×(1 - δ(α′A + β′L))× (1 - γ′N) + γ′B × N; Among them, I syn For input, a clear synthetic image; I deg The output is the degraded image; A, L, and N are the aging mapping parameters, illumination mapping parameters, and noise mask parameters extracted from the statistics of real ancient book samples, respectively; B is the background fill value, used to simulate the realistic appearance of the hole area; δ is the attenuation adjustment factor.

4. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 3, characterized in that, Step S4: Design a multi-dimensional quality assessment and reverse feedback mechanism to automatically evaluate the quality of the degraded samples, and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples, specifically including: Step S41: Image fidelity metrics: Structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) are used to measure the similarity between the generated degraded samples and the real ancient books in terms of texture, lighting, and edges. Step S42: Semantic consistency index: Introduce the cross-modal semantic matching index CLIPScore to evaluate the matching degree between the visual features of the generated samples and the semantic space of the text; Step S43: Structural annotation accuracy indicators: The accuracy of automatically annotated regions and structures is evaluated by using OCR recognition accuracy and structural intersection-union ratio (IoU). Step S44: Define the overall quality scoring function: Q=w1×SSIM+w2×PSNR+w3×CLIPScore+w4×OCRAcc+w5×SioU; Where w1~w5 are empirical weights, satisfying ∑w i =1, i=1-5; OCRACcc is the OCR recognition accuracy; SioU is the intersection-union ratio; Set minimum threshold Q min When the degraded sample is below the threshold, the process proceeds to step S3 for parameter adjustment. Specifically, this involves triggering an automatic parameter adjustment process, including: performing a constrained parameter search or gradient approximation update based on Q for (α,β,γ), updating the mapping weights and threshold set of the degradation control function, and regenerating the degraded sample until Q≥Q. min .

5. The method for generating high-fidelity annotations for multilingual ancient books driven by reverse synthesis according to claim 4, characterized in that, Step S5: Using a lightweight object detection and OCR recognition network, semantic-level annotation is performed on the final degraded samples. Combined with readback consistency detection, region IOU correction, and multi-model consistency determination, the final standardized annotation result is generated, specifically including: Step S51: Input the degraded sample into a lightweight object detection system to automatically identify the structural regions of the main text area, title area, annotation area, illustration area, and stamp area, and generate corresponding detection boxes and label information; then input the degraded sample into an OCR recognition network to obtain the labeled text content, language attributes, and degradation parameters corresponding to the image; Step S52: Perform OCR recognition on the annotated text content again to obtain the reread result; calculate the edit distance and semantic matching degree between the two to obtain the reread consistency score. When the reread consistency score is lower than the preset threshold, trigger the local re-recognition or re-annotation operation. Step S53: Calculate the Intersection over Union (IOU) between each pair of detection boxes. When the IOU exceeds the threshold, it is determined that the two boxes are duplicate or overlapping, and then the boxes are automatically merged, the boundaries are smoothed, and the coordinates are adjusted. Step S54: Call the main detection model and the validation model in parallel for the same page image and perform independent annotation tasks respectively; compare the three key results output by the two models: structure category, boundary coordinates and text content; when the difference exceeds the threshold, take the high confidence result as the main output.

6. A reverse synthesis-driven multilingual ancient book high-fidelity annotation generation system, characterized in that, Includes the following modules: The data acquisition module is used to collect multilingual ancient text data from multiple sources; through language segmentation, noise removal and structural standardization, an OCR corpus covering multiple languages ​​is constructed. The rendering module is used to perform typesetting modeling and visual rendering based on the unique writing rules of each language, using multilingual structural modeling algorithms, and to achieve preliminary structural-level annotation based on the spatial hierarchy of the main text, title, annotations, seals, and illustrations. The visual simulation and degradation module is used to construct degradation models for degradation samples. Through multi-parameter mapping and combination control, it can accurately adjust the degradation type, intensity and distribution, so that the degradation samples are highly consistent with real ancient books in terms of texture, brightness and structure. The quality assessment and feedback module is used to design a multi-dimensional quality assessment and feedback mechanism to automatically assess the quality of the degraded samples and perform dynamic self-correction and closed-loop optimization based on the assessment results to obtain the final degraded samples. The annotation and structure recognition module is used to perform semantic-level annotation on the final degraded sample using a lightweight object detection and OCR recognition network, and to generate the final standardized annotation result by combining readback consistency detection, region IOU correction and multi-model consistency judgment.

7. A reverse synthesis-driven multilingual ancient book high-fidelity annotation generation device, characterized in that, It includes one or more electronic devices, wherein the one or more electronic devices are used to implement the method of any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5.

Images