Ancient book text restoration method based on generative artificial intelligence and related equipment

Abstract

The application discloses a kind of ancient book text restoration method based on generative artificial intelligence and related equipment, it is related to artificial intelligence and ancient book text restoration technical field, method includes: to the obtained ancient book image of restoration is carried out layout analysis and pretreatment;Deterioration area in the image after pretreatment is carried out semantic level reconstruction, obtain enhanced image, and extract recognizable text and missing area position information from enhanced image;According to recognizable text and missing area position information, obtain the context text of missing area;Context text is input into ancient book special language model, generates multiple candidate repair texts;Obtain the multimodal fusion confidence of each candidate repair text;From multiple candidate repair texts, the optimal repair text is screened out, and the optimal repair text is filled into missing area, to obtain repaired text;The application can ensure that repair result is consistent with historical language specification and original image form.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and ancient book text restoration technology, and in particular to an ancient book text restoration method and related equipment based on generative artificial intelligence. Background Technology

[0002] With the revival of outstanding traditional culture and the deepening of digital humanities research, the digitization and intelligent restoration technologies of ancient books are increasingly becoming a crucial link in cultural heritage protection. In the long process of preservation and transmission, ancient books and documents generally suffer from degradation phenomena such as ink fading, insect damage, paper aging, and changes in character form, resulting in a large amount of text being difficult to identify or completely missing. Traditional methods of restoring ancient books mainly rely on manual collation by experts, which is not only time-consuming and labor-intensive but also limited by the experts' experience and subjective judgment, making it difficult to meet the practical needs of large-scale ancient book collation and digitization projects. Therefore, there is an urgent need for a technological solution that can efficiently and accurately automate the restoration of ancient book texts.

[0003] In recent years, image processing and optical character recognition (OCR) technologies have been introduced into the field of ancient book restoration, attempting to restore incomplete text through image enhancement and character pattern recognition. Meanwhile, existing deep learning-based methods for ancient book restoration often employ convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to perform end-to-end modeling of ancient book images, attempting to directly reconstruct text from damaged images. These methods leverage the powerful feature extraction capabilities of neural networks to learn the mapping relationship between degraded regions and complete text at the image pixel level, providing an automated technical path for ancient book restoration.

[0004] However, traditional OCR systems suffer from significant accuracy drops when faced with common problems in ancient books, such as blurred ink, insect damage, and distorted characters, making it difficult to effectively restore the original text. CNN or RNN-based methods typically focus only on local pixel information, lacking an understanding of contextual semantics and the rules of classical Chinese language, resulting in significant deficiencies in semantic coherence and historical character usage. Furthermore, most model training relies on large amounts of high-quality labeled data, but labeling ancient books is extremely costly, and data scarcity severely restricts the improvement of model generalization capabilities. Generative artificial intelligence, particularly the rise of Large Language Models (LLM), has provided a new technical path for the restoration of ancient texts. After pre-training on massive amounts of text, LLM possesses powerful language modeling and contextual reasoning capabilities, able to reasonably infer missing characters based on incomplete context, achieving high-precision semantic completion. However, general-purpose large language models are not optimized for the characteristics of ancient texts, and have limited understanding of special linguistic phenomena such as classical Chinese grammar, variant characters, and phonetic loan characters. Directly applying them to the restoration of ancient books can easily lead to temporal misalignment or semantic bias. Meanwhile, language models at the pure text level cannot utilize the remaining stroke directions and character structure information in ancient book images, resulting in the restoration results lacking alignment constraints with the original image in terms of visual form. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing technologies, specifically providing a method and related equipment for ancient text restoration based on generative artificial intelligence, as detailed below: 1) In a first aspect, the present invention provides a method for restoring ancient texts based on generative artificial intelligence, the specific technical solution of which is as follows: The obtained images of ancient books to be restored are subjected to layout analysis and preprocessing to obtain preprocessed images; The degraded region in the preprocessed image is semantically reconstructed by a denoising diffusion probability model to obtain an enhanced image, and the recognizable text and missing region location information are extracted from the enhanced image. Based on the recognizable text and the location information of the missing area, obtain the contextual text of the missing area; Input the contextual text into a language model specifically designed for ancient books to generate multiple candidate texts for repair. Multiple candidate repair texts are aligned with the enhanced image. The text features of the candidate repair texts and the structural features of the corresponding positions in the enhanced image are weighted and fused through an attention mechanism to obtain the multimodal fusion confidence of each candidate repair text. Based on the multimodal fusion confidence of each candidate repair text, the optimal repair text is selected from multiple candidate repair texts, and the optimal repair text is filled into the missing area to obtain the repaired text. The repaired text is selected as the final repair result.

[0006] The beneficial effects of the ancient book text restoration method based on generative artificial intelligence provided by this invention are as follows: By performing layout analysis and preprocessing on the images of ancient books to be restored, text regions, image regions, and missing regions are accurately identified and segmented, providing a reliable image foundation for subsequent restoration. A denoising diffusion probability model is used to perform semantic-level reconstruction of degraded regions in the preprocessed images. Combined with global structural features extracted by a visual Transformer to guide the restoration process, this enables content-aware restoration of complex degraded regions such as ink smudges and insect damage, effectively overcoming the shortcomings of traditional convolutional neural networks that rely solely on local pixel information and lack global semantic understanding. By obtaining the contextual text of the missing regions based on identifiable text and the location information of the missing regions, and inputting this contextual text into a language model specifically optimized for ancient Chinese, multiple candidate restoration texts are generated. Utilizing a language model optimized specifically for Classical Chinese, accurate understanding and reasonable inference of Classical Chinese grammar, variant characters, and phonetic loan characters are achieved at the semantic level, effectively solving the problems of temporal misalignment and semantic bias that easily arise when general-purpose language models are directly applied to the restoration of ancient books. By aligning multiple candidate repair texts with the enhanced image, an attention mechanism is used to weight and fuse the text features of the candidate repair texts with the structural features of the corresponding positions in the enhanced image, thereby obtaining the multimodal fusion confidence of each candidate repair text. This allows for the selection of the repair text that is optimal in both semantic logic and visual morphology from multiple candidates, ensuring that the repair result conforms to historical language norms and is consistent with the original image morphology.

[0007] 2) Secondly, the present invention also provides an ancient book text restoration system based on generative artificial intelligence, the specific technical solution of which is as follows: It includes an image processing module, a reconstruction and extraction module, a context acquisition module, a generation module, a confidence acquisition module, a filtering and input module, and a confirmation module; The image processing module is used to: perform layout analysis and preprocessing on the acquired images of ancient books to be restored, and obtain the preprocessed image; The reconstruction and extraction module is used to: perform semantic-level reconstruction of the degraded regions in the preprocessed image using a denoising diffusion probability model to obtain an enhanced image, and extract recognizable text and missing region location information from the enhanced image; The context acquisition module is used to: obtain the context text of the missing region based on the recognizable text and the location information of the missing region; The generation module is used to: input the context text into the ancient book-specific language model and generate multiple candidate repair texts; The confidence acquisition module is used to: align multiple candidate repair texts with the enhanced image, and use an attention mechanism to perform weighted fusion of the text features of the candidate repair texts and the structural features of the corresponding positions in the enhanced image to obtain the multimodal fusion confidence of each candidate repair text; 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so as to enable the electronic device to implement any of the above-mentioned methods for restoring ancient texts based on generative artificial intelligence.

[0008] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for restoring ancient texts based on generative artificial intelligence.

[0009] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below: Figure 1 This is a flowchart illustrating a method for restoring ancient texts based on generative artificial intelligence, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an ancient text restoration system based on generative artificial intelligence, according to an embodiment of the present invention. Detailed Implementation

[0011] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0012] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0013] like Figure 1 As shown in the figure, an ancient text restoration method based on generative artificial intelligence according to an embodiment of the present invention includes the following steps: S1. Perform layout analysis and preprocessing on the acquired images of the ancient books to be restored to obtain preprocessed images, including: S10. Perform layout analysis on the image of the ancient book to be restored, detect the text areas, image areas, and damaged areas in the image, and segment and label the text areas, image areas, and damaged areas to obtain the location information of the damaged areas. Specifically: The acquired images of the ancient books to be restored undergo layout analysis, employing a deep learning-based document image segmentation model to process the entire image. The images are input into a pre-trained visual Transformer model, which divides the image into multiple fixed-size image blocks and extracts global contextual features for each block. Based on these extracted global contextual features, the model classifies each pixel or image block to determine its region category. During layout analysis, text regions, image regions, and damaged regions are detected in the images. Text regions refer to areas containing readable text; image regions refer to areas of the ancient book containing non-textual content such as illustrations and seals; damaged regions refer to areas where the original text is missing or illegible due to ink smudges, insect damage, paper aging, etc. The model labels each location in the image as either a text region, image region, or damaged region using pixel-level or region-level classification.

[0014] The process involves segmenting and labeling the detected text regions, image regions, and incomplete regions. Segmentation and labeling refers to marking the various regions detected by the layout analysis as closed boundaries in the image, making each region an independent sub-region. For text regions, segmentation and labeling generate region boundaries containing complete lines of text; for image regions, segmentation and labeling generate region boundaries containing illustrations or seals; for incomplete regions, segmentation and labeling generate region boundaries containing damaged or missing parts. The results of segmentation and labeling are recorded in the form of region masks, where the value of each pixel in the mask represents the region category to which that pixel belongs. The location information of incomplete regions is obtained from the results of segmentation and labeling. The location information includes the coordinate range of the incomplete region in the image, specifically the x and y coordinates of the top-left pixel of the incomplete region, and the width and height of the incomplete region. The location information of the incomplete region is recorded as a quadruple data set, used in subsequent steps to locate the specific position of the missing text in the image.

[0015] In the field of document image segmentation, commonly used deep learning models include UNet and its variants based on encoder-decoder architecture, fully convolutional networks (FCN), DeepLab series, YOLO series and other object detection models, as well as models based on Transformer architecture such as Swin Transformer, DocSAM, HisDocDETR and others. Taking a document image segmentation model based on visual Transformer as an example, its training process includes: First, constructing a training dataset by selecting samples from a large number of ancient book images, covering different dynasties, fonts, storage formats, and rotation directions. Then, pixel-level segmentation and annotation are performed on text regions, image regions, and incomplete regions in the images to generate corresponding segmentation masks. Next, the ancient book images are input into the visual Transformer network, where the images are divided into multiple fixed-size image blocks. Each image block is linearly projected and positionally encoded before being input into a multi-layer Transformer encoder, which extracts global contextual features through a self-attention mechanism. The model outputs a segmentation prediction map with the same size as the input image through the decoder, where each pixel is classified as a text region, image region, or incomplete region. During training, the difference between the predicted segmentation map and the true segmentation mask is calculated using the cross-entropy loss function or the Dice loss function, and the model parameters are iteratively updated using the backpropagation algorithm. After training, the segmentation accuracy of the model is evaluated using a reserved test set, and hyperparameters are adjusted or training data is increased based on the evaluation results until the model achieves the expected segmentation performance.

[0016] S11. Preprocess the segmented and labeled image of the ancient book to be restored. The preprocessing includes image denoising and adaptive contrast enhancement of the text regions to obtain the preprocessed image. Specifically: Preprocessing is performed on the segmented and labeled images of the ancient book to be restored. Preprocessing is only performed on the text regions in the segmentation and labeling results; image regions and damaged areas are not included in the preprocessing. Specifically: ① Image denoising is performed on the text regions. Gaussian filtering is used to process the text regions. For each pixel in the text region, Gaussian filtering takes a Gaussian weighted average of the gray values ​​of all its surrounding pixels and replaces the original gray value of the center pixel with the weighted average. The weights of the Gaussian filter are calculated based on the Gaussian function, with neighboring pixels closer to the center pixel receiving higher weights. Through Gaussian filtering, random noise points in the text region are smoothly eliminated, while the edge structure of the character strokes is preserved. After image denoising, noise interference such as spots and lines in the text region is effectively suppressed. ② Adaptive contrast enhancement is performed on the denoised text regions. Adaptive contrast enhancement uses a contrast-limited adaptive histogram equalization algorithm. This algorithm divides the text region into multiple local sub-blocks and calculates the distribution histogram of pixel gray values ​​for each local sub-block. Based on the histogram distribution characteristics of each local sub-block, a contrast stretching mapping function is calculated for each sub-block to make the histogram distribution more uniform. When calculating the mapping function, a height limit threshold is set for the histogram to prevent excessive amplification of local contrast, which could lead to noise amplification. After processing each local sub-block, bilinear interpolation is used to eliminate boundary effects between sub-blocks, resulting in a smooth transition of contrast enhancement across the entire text area. After adaptive contrast enhancement, the distinction between text strokes and the background, which were previously low in contrast due to fading or uneven lighting, is improved.

[0017] The text regions, after image denoising and adaptive contrast enhancement, are merged with the unprocessed image regions and incomplete regions according to the segmentation annotation boundaries to obtain the preprocessed image. In the preprocessed image, the image quality of the text regions is improved, while the original image regions and incomplete regions remain unchanged.

[0018] S2. Semantic-level reconstruction of degraded regions in the preprocessed image is performed using a denoising diffusion probability model to obtain an enhanced image. Recognizable text and location information of missing regions are extracted from the enhanced image, including: S20. The trained visual Transformer network is used to extract the global structural features of the preprocessed image. Specifically: The preprocessed image is input into a trained visual Transformer network. The trained visual Transformer network divides the preprocessed image into multiple fixed-size image patches, for example, 16 pixels by 16 pixels. Each image patch is flattened into a one-dimensional vector and linearly projected to become its embedding vector. A positional encoding is added to the embedding vector of each image patch, recording its spatial location information in the original image, including its horizontal and vertical positions. The embedding vector sequence with positional encoding preserves the spatial arrangement between image patches. This sequence is then input into a multi-layer Transformer encoder. The self-attention mechanism in the Transformer encoder processes the embedding vector sequence, enabling each image patch's embedding vector to integrate information from all other image patches in the sequence during updates. Through iterative processing by the multi-layer Transformer encoder, the embedding vector of each image patch incorporates contextual information from all locations within the entire image. Global structural features of the preprocessed image are extracted from the output of the multi-layer Transformer encoder. Global structural features are represented in the form of feature maps. Each position in the feature map corresponds to a local region in the original image, and the feature vector at each position in the feature map encodes the global dependency between that local region and other regions of the entire image.

[0019] The training process of the visual Transformer network is as follows: First, a training dataset is constructed by selecting samples from a large number of ancient book images, covering different dynasties, fonts, storage formats, and rotation directions. Pixel-level segmentation and annotation are performed on text regions, image regions, and incomplete regions in the images to generate corresponding segmentation masks. During training, the ancient book images are input into the visual Transformer network. The images are divided into multiple fixed-size image blocks. Each image block is linearly projected and positionally encoded before being input into a multi-layer Transformer encoder. The multi-head self-attention mechanism in the encoder enables the feature representation of each image block to integrate information from all other image blocks in the entire image, thereby extracting features covering the global structure of the image. For the segmentation task, the feature map output by the encoder is restored to the size of the input image by the decoder or linear segmentation head, outputting a segmentation prediction map. Each pixel in the prediction map is classified as a text region, image region, or incomplete region. During training, the cross-entropy loss function or Dice loss function is used to calculate the difference between the predicted segmentation map and the true segmentation mask, and the model parameters are iteratively updated using the backpropagation algorithm. The optimizer typically uses AdamW, combined with a learning rate scheduling strategy. After training is complete, the segmentation accuracy of the model is evaluated using a reserved test set, and the hyperparameters are adjusted or training data is increased based on the evaluation results until the model achieves the expected segmentation performance.

[0020] S21. The global structural features are used as conditional information to input the denoising diffusion probability model. The denoising diffusion probability model is used to perform step-by-step denoising on the degraded regions in the preprocessed image. At each denoising stage, content-aware restoration is performed on the degraded regions based on the global structural features to generate the restored image regions. The specific implementation process is as follows: S210. The global structural features extracted by the trained visual Transformer network are used as conditional information input into the denoising diffusion probability model. The conditional information is input in the following way: at each denoising stage of the inverse denoising process of the denoising diffusion probability model, the intermediate layer features of the current stage denoising network are concatenated with the global structural features, or fused through a cross-attention mechanism. By using the global structural features as conditional information, the denoising diffusion probability model can refer to the overall layout and spatial structure of the preprocessed image when generating the repaired content, ensuring that the repaired content maintains consistency with the original image in terms of layout and spatial position.

[0021] S211. The denoising diffusion probability model performs a reverse denoising process starting from the pure noise image. The pure noise image is an image of the same size as the preprocessed image, where the value of each pixel is randomly sampled from a standard normal distribution. The reverse denoising process consists of multiple denoising stages, the total number of which is consistent with the preset diffusion steps during training, typically set to hundreds to thousands of steps. Each denoising stage corresponds to a reverse step in the forward denoising process; that is, in the forward denoising process, the clear image becomes a noisier image after the t-th step of denoising; in the reverse denoising process, the model starts from the pure noise image at step T and progressively performs the reverse operation from step T to step 1. In each denoising stage, the denoising diffusion probability model performs a denoising operation on the current image, gradually removing noise from the image, causing the current image to converge towards a clear image.

[0022] S212. In each denoising stage, the denoising diffusion probability model performs content-aware restoration of degraded regions in the preprocessed image based on the input global structural features. The specific implementation of content-aware restoration is as follows: In each denoising stage, the denoising diffusion probability model fuses the current image with the global structural features. The fusion method involves adjusting the global structural features to the same spatial size as the current image through spatial transformation, and then stitching them together along the channel dimension. The stitched features are used as input to the denoising network. The denoising network determines the position and expected content form of the degraded region in the entire image based on the overall image layout and spatial structure information recorded in the global structural features. Based on this determination, the denoising diffusion probability model performs targeted generation of content in the degraded region during the denoising process. The specific method of targeted generation is as follows: During each denoising update step, for pixel positions belonging to the degraded region, the model adjusts the pixel value update direction based on the guidance of the global structural features, ensuring that the pixel content gradually generated in the degraded region is consistent with the expected content indicated by the global structural features; for pixel positions in non-degraded regions, the model maintains their original structural information unchanged during denoising.

[0023] S213. After iterative processing through all denoising stages, the denoising diffusion probability model outputs the restored image region. The restored image region corresponds to the location of the degraded region in the preprocessed image. In the restored image region output by the denoising diffusion probability model, the areas that were originally degraded due to ink smudges, insect damage, or paper aging have been filled with newly generated pixel content. The restored image region contains the complete character structure and texture content generated after semantic-level reconstruction based on global structural features.

[0024] The denoising diffusion probabilistic model is a generative model that achieves image generation and restoration through a forward denoising process and a reverse denoising process. The forward denoising process progressively adds Gaussian noise to a clear image through a multi-step Markov chain. After a predetermined number of diffusion steps, the clear image evolves into a purely random noise map. The reverse denoising process starts from the pure noise map and iteratively removes noise through multiple steps. In each step, the model predicts the clear image corresponding to the current noise map and updates the image based on the prediction result. In image restoration tasks, the reverse denoising process can be guided by conditional information, allowing the model to refer to conditional information at each denoising step, thereby generating a restoration result that meets the expected content. The training process of the denoising diffusion probabilistic model learns the predictive ability of reverse denoising; that is, given a noise map after any number of denoising steps, the model can predict the corresponding clear image or predict the distribution of the added noise.

[0025] S22. The repaired image region is fused with the non-degraded region in the preprocessed image to obtain the enhanced image, specifically: The non-degraded regions in the preprocessed image are identified. These non-degraded regions are image areas not marked as degraded, including complete text and image regions. The restored image region output from the denoising diffusion probability model is aligned with the non-degraded regions in the preprocessed image according to their respective positions within the image. The positions of the restored image regions correspond exactly to the positions of the degraded regions in the preprocessed image, while the positions of the non-degraded regions remain unchanged. The aligned restored image regions and the non-degraded regions are merged at the pixel level to generate the enhanced image. In the enhanced image, the original degraded regions are replaced by the restored image regions, while the non-degraded regions retain their original preprocessed state. The overall clarity and readability of the enhanced image are higher than those of the preprocessed image.

[0026] S23. Perform optical character recognition on the enhanced image to extract recognizable text from the enhanced image, and extract the location information of the missing area from the location of the missing area in the enhanced image based on the location information of the missing area obtained from the layout analysis. Specifically: S230. Perform optical character recognition (OCR) on the enhanced image. The OCR model detects and recognizes all text regions in the enhanced image, matches each detected character image block with standard characters in the character library, and outputs the corresponding character code. The OCR model outputs the recognition result, which includes the code of each recognized character and its position coordinates in the enhanced image.

[0027] Optical character recognition (OCR) models are primarily based on the CRNN architecture or its improved variants. CRNN stands for Convolutional Recurrent Neural Network, which consists of a concatenated convolutional neural network, a recurrent neural network, and a connection-time classification loss function. During model training, a dataset containing a large number of image-text pairs is first constructed. Images undergo preprocessing operations such as normalization and size unification. The preprocessed images are then input into the convolutional neural network, which extracts visual feature maps from the images through multiple convolutional and pooling operations. These feature maps are compressed into feature sequences, with each time step corresponding to a local region in the original image. The feature sequences are then input into the recurrent neural network, which typically employs a bidirectional long short-term memory network to model the context of the feature sequences, capturing the dependencies between positions within the sequence and outputting the predicted probability distribution for each character category at each time step. Finally, the predicted probability distribution output by the recurrent neural network is input into the connection-time classification module. The connection-time classification loss function calculates the difference between the predicted results and the actual text labels without requiring a one-to-one correspondence between the input and output label sequences. A dynamic programming algorithm is used to search for the optimal label alignment path and calculate the loss value. The loss value calculated using the connection-time classification loss function is used to iteratively update all parameters of the convolutional neural network and the recurrent neural network through the backpropagation algorithm until the model's recognition accuracy on the validation set converges. For ancient text recognition tasks, the training process typically involves pre-training with synthetic ancient text image data, followed by fine-tuning with a small amount of real ancient text labeled data to improve the model's recognition accuracy in ancient text scenarios.

[0028] S231. Extract recognizable text from the enhanced image based on the recognition results of optical character recognition. Recognizable text refers to the text content that the optical character recognition model successfully recognizes and outputs character codes for. Each recognizable text corresponds to a specific character position in the enhanced image.

[0029] S232. Obtain the location information of the missing area recorded during the layout analysis stage. The location information of the missing area has been obtained and stored in advance during the layout analysis stage, which records the coordinate range and size of the missing area in the image.

[0030] S233. Based on the location information of the missing region, locate the region corresponding to the location of the missing region in the enhanced image. Since the enhanced image and the preprocessed image are completely identical in spatial size and coordinate system, the location information of the missing region is also valid in the enhanced image.

[0031] S234. Extract the location information of the missing region from the region corresponding to the location of the missing region in the enhanced image. The location information of the missing region includes the specific coordinates of each missing character in the missing region in the enhanced image. The extraction method is as follows: within the coordinate range of the missing region, the positions of characters that could not be recognized by optical character recognition are recorded one by one as the location information of the missing region. The location information of the missing region is recorded in the form of a coordinate list, where each item in the list corresponds to the position of a missing character in the enhanced image.

[0032] Global structural features refer to the feature representations extracted from the entire image that reflect the overall layout and spatial structure of the image. In ancient book image processing, global structural features include information such as the overall arrangement of text regions, the relative positions between lines, the spacing between characters, and the spatial distribution of image and text regions. Degraded regions refer to image areas in ancient books where the original text content is damaged or unrecognizable due to ink smudges, insect damage, paper aging, etc. Degraded regions appear as blurred, missing, or distorted local areas in the image, where the original character structure information has been partially or completely lost. Semantic-level reconstruction refers to the process of not only restoring the pixel color and texture information of degraded regions during image restoration, but also performing content-aware filling and restoration of degraded regions based on the global semantic content of the image. Semantic-level reconstruction requires that the restored region visually conforms to the overall style of the surrounding area and maintains consistency with the overall semantics of the image in terms of content. Optical character recognition (OCR) is a technology that automatically converts visual text in an image into computer-readable text data. OCR technology detects text regions in an image, performs shape analysis and pattern matching on each character, and outputs the corresponding character code. Optical character recognition (OCR) can extract printed or handwritten text from scanned documents, photographs, and other images. Missing region location information refers to the specific coordinates of areas in ancient book images where text content is missing due to degradation. This information includes the x and y coordinates of the top-left pixel of the missing region, as well as the width and height of the missing region, recorded as a quadruple.

[0033] S3. Based on the recognizable text and the location information of the missing region, obtain the context text of the missing region, including: S30. Based on the location information of the missing region, determine the direction of the text line to which the missing region belongs in the enhanced image, and determine the relative position of the missing region within the text line. Specifically: The missing region's location information is obtained, recording its coordinate range in the enhanced image, including the x and y coordinates of the top-left pixel, as well as its width and height. Based on this information, the missing region is located in the enhanced image, and the coordinates of the adjacent recognizable characters are obtained. These coordinates are provided by the optical character recognition (OCR) step; each recognizable character has corresponding coordinates in the enhanced image. The direction of the text line to which the missing region belongs is determined based on the coordinates of the missing region and its adjacent recognizable characters. This direction is determined by comparing the coordinate distribution of multiple recognizable characters within the same text line: if the x-coordinates of characters in the same line increase sequentially while their y-coordinates remain relatively constant, the text line is considered horizontally arranged; if the y-coordinates increase sequentially while their x-coordinates remain relatively constant, the text line is considered vertically arranged. Based on the characteristics of ancient books, the text line direction can also be pre-set using the layout results from the page analysis stage. After determining the text line direction, all recognizable characters in the text line to which the missing region belongs are sorted according to their text line direction, forming an ordered sequence. Based on this ordered sequence, the relative position of the missing region within its corresponding line of text is determined. The relative position is determined by counting the number of recognizable characters before and after the missing region in the ordered sequence, while also recording the sequence number of the missing region within the entire line of text.

[0034] S31. From the recognizable text, extract a predetermined number of recognizable characters preceding the missing region from the text line containing the missing region as the preceding text, and extract a predetermined number of recognizable characters following the missing region from the text line containing the missing region as the following text. Specifically: S310. Obtain the preset quantity parameter. The preset quantity is a pre-set positive integer used to specify how many recognizable characters to extract before and after the missing area as context. The value of the preset quantity is determined based on the requirements of the ancient text-specific language model for the length of the input text and the requirements for repair accuracy.

[0035] S311. Based on the text line direction, extract a preset number of recognizable characters from the text line containing the missing area, preceding the missing area, as the preceding text. During extraction, select characters one by one from the text immediately adjacent to the missing area in reverse order of the text line direction until the preset number of recognizable characters are extracted. The text order in the preceding text maintains the original order of the text line direction.

[0036] S312. Based on the text line direction, extract a preset number of recognizable characters from the text line containing the missing area, which are located after the missing area, as the following text. During extraction, select characters one by one from the text immediately adjacent to the missing area, following the text line direction, until the preset number of recognizable characters are extracted. The text order in the following text maintains the original text line direction order.

[0037] S32. Concatenate the preceding and following texts in the order of the text lines to obtain the context text of the missing area; when the number of recognizable characters before the missing area in the text line to which the missing area belongs is less than a preset number, extract the last few recognizable characters from the text line preceding the missing area and add them to the preceding text; when the number of recognizable characters after the missing area in the text line to which the missing area belongs is less than a preset number, extract the first few recognizable characters from the text line following the missing area and add them to the following text. The specific implementation process is as follows: S320. Concatenate the preceding and following texts in the order of the text lines. When concatenating, place the preceding text first, then place the following text last, leaving a position mark between them to represent the missing area, forming the context text of the missing area.

[0038] S321. Determine if the number of recognizable characters preceding the missing region in the text line to which the missing region belongs is less than a preset number. If the number of recognizable characters is greater than or equal to the preset number, directly use the extracted preceding text; if the number of recognizable characters is less than the preset number, extract the last few recognizable characters from the preceding text line to supplement the preceding text. During supplementation, start extracting recognizable characters from the end of the preceding text line backwards until the sum of the supplemented characters and the original recognizable characters reaches the preset number. The supplemented preceding text is rearranged according to the text line direction, with the last characters of the preceding text line preceding the original preceding text in the corresponding line.

[0039] S322. Determine if the number of recognizable characters following the missing region in the text line to which the missing region belongs is less than a preset number. If the number of recognizable characters is greater than or equal to the preset number, directly use the extracted following text; if the number of recognizable characters is less than the preset number, extract the first few recognizable characters from the next text line to which the missing region belongs and add them to the following text. When adding, start extracting recognizable characters from the beginning of the next text line and continue until the sum of the added characters and the original recognizable characters reaches the preset number. The following text after adding is rearranged according to the order of the text lines, with the first character of the next text line following the original following text in the corresponding line.

[0040] S323. After the supplementation process, the complete preceding text and the complete following text are concatenated in the order of the text lines to obtain the final context text. The text arrangement order in the context text is consistent with the actual reading order of the ancient book and is used for subsequent input into the ancient book-specific language model.

[0041] The text line direction refers to the arrangement direction of text lines within the layout of an ancient book image. Ancient books have two basic text line directions: horizontal and vertical. Vertical arrangement is the common top-to-bottom, right-to-left writing direction in ancient books, while horizontal arrangement is left-to-right. Determining the text line direction is crucial for correctly judging the reading order and character arrangement logic of the text lines in the missing area.

[0042] In this context, relative position refers to the location of the missing region within its corresponding text line relative to the start and end positions of the entire line. Relative position is described by the number of recognized characters before and after the missing region, or by the missing region's index or percentage position within the text line. Relative position is used to determine the range within a text line when extracting preceding and following text.

[0043] In this context, the preceding text refers to the identifiable text preceding the missing area in the lines of ancient texts. The preceding text is ordered according to the reading direction of the lines, forming a semantic sequence preceding the missing area. The preceding text provides contextual information preceding the missing area, assisting the language model in inferring the missing text.

[0044] In this context, the following text refers to the identifiable text content following the missing area in a line of text in an ancient book. The following text is ordered according to the reading direction of the lines, forming a semantic sequence following the missing area. The following text provides contextual information following the missing area, assisting the language model in inferring the missing text.

[0045] The context text is a complete text sequence composed of preceding and following text, arranged in line-by-line order. It includes recognizable text within a certain range before and after the missing area, forming a continuous text string that is input into a language model specifically designed for ancient texts to generate candidate repair text.

[0046] S4. Input the context text into the ancient text-specific language model to generate multiple candidate texts for repair. The specific implementation process is as follows: S40. Construct a language model specifically for ancient texts. The process of constructing a language model specifically for ancient texts includes the following sub-steps: The first sub-step involves selecting a base model. This base model employs a large language model with a Transformer decoder architecture and 1.8 billion parameters. The selection of 1.8 billion parameters is based on the scarcity of high-quality labeled data in the field of ancient texts. Too many parameters can easily lead to model overfitting, while too few parameters would make it difficult to capture the rich linguistic and cultural knowledge within ancient texts. The base model chosen is an open-source large language model that has already been pre-trained on a large-scale corpus of modern Chinese.

[0047] The second sub-step involves constructing the training corpus, which consists of a large-scale, high-quality collection of ancient texts, including full-text data of classic works such as the *Siku Quanshu* and the *Yongle Encyclopedia*. Original text data is collected from various digitized resources of ancient books, covering different categories of literature, including classics, history, philosophy, and literature, and spanning all historical periods from the pre-Qin period to the Ming and Qing dynasties. The collected original texts undergo data cleaning to remove optical character recognition errors, duplicate paragraphs, modern annotations, and other noise. The cleaned texts are then processed to unify traditional and simplified Chinese characters, ensuring the model is compatible with both. Finally, the processed texts are categorized and archived according to source, historical period, and document type, forming a structured training corpus.

[0048] The third sub-step involves further training the base model using the constructed training corpus, building upon its existing parameters. Pre-training continues using a self-supervised learning approach, randomly masking portions of text from the input classical Chinese texts. The model is then required to predict the actual text at the masked locations based on the context. The model's predictions are compared with the actual text, and the loss is calculated. This loss function guides the update of the model parameters. By iteratively performing the masked language modeling task on a large-scale classical Chinese corpus, the model gradually learns the vocabulary distribution, grammatical structure, and historical word usage patterns of classical Chinese.

[0049] The fourth sub-step involves supervised fine-tuning using specialized labeled data for ancient text restoration. This specialized labeled data is constructed as follows: complete sentences are selected from the ancient text, and degradation conditions such as ink smudges and insect damage are manually simulated. Some text is replaced with missing markers, and the original text is labeled as the restoration target. The text with missing markers is input into the pre-trained model. The model generates predicted text at the missing marker locations. The model's predicted text is compared with the manually labeled original text, and the loss is calculated. The model parameters are updated using the backpropagation algorithm based on the loss function. After multiple rounds of iterative training, the model gradually learns to accurately infer missing text based on contextual semantics and historical character usage norms.

[0050] The fifth sub-step involves evaluating the model using a reserved test dataset, which is not used in the training process. The evaluation metric is the repair accuracy, which is the proportion of the original text that the model predicts. If the evaluation result does not reach the preset threshold, the training parameters, such as the learning rate and the number of training epochs, are adjusted, and supervised fine-tuning is performed again until the model's repair accuracy on the test dataset reaches the preset requirement.

[0051] S41. Obtain the context text of the missing region generated in step S32. The context text contains a recognizable sequence of characters concatenated before and after the missing region according to the text line direction, with the positional markers of the missing region retained in the sequence. The context text is passed as input to the ancient text-specific language model. The ancient text-specific language model performs semantic encoding on the input context text. The model converts each character in the context text into a semantic vector representation through a multi-layer Transformer network. During the encoding process, the model's self-attention mechanism ensures that the semantic vector of each character incorporates information from all other characters in the context text, thereby gaining an understanding of the overall semantics of the context surrounding the missing region. Based on the encoded semantic information, the ancient text-specific language model performs autoregressive text generation at the positional markers of the missing region. The autoregressive text generation predicts the characters at each position in the missing region one by one from left to right. When predicting each position, the model calculates the probability distribution of all characters and words in the model based on the overall semantics of the generated preceding characters and the original context text, selecting the characters with the highest probabilities as candidate characters for that position. The ancient text-specific language model retains multiple generation paths simultaneously during the generation process using a beam search algorithm, with each path corresponding to a candidate text for repair. The beam search algorithm retains the paths with the highest probability at each generation step, ultimately outputting a predetermined number of complete candidate texts for repair. Each candidate text is accompanied by a generation confidence score assigned by the model, and the candidate texts are arranged in descending order of their generation confidence scores.

[0052] S5. Align multiple candidate repair texts with the enhanced image, and use an attention mechanism to weightedly fuse the text features of the candidate repair texts with the structural features of corresponding positions in the enhanced image to obtain the multimodal fusion confidence of each candidate repair text, including: S50. For each candidate character in the candidate repair text, determine the corresponding filling position of the candidate character in the enhanced image based on the missing region location information, and use the semantic embedding vector of the candidate character as the text feature of the candidate character. Specifically: For each candidate character in the candidate repair text, its position index is obtained. The candidate characters are arranged from left to right, and each candidate character has a unique position index. The filling position of the candidate character in the enhanced image is determined based on the missing region location information. The missing region location information records the spatial coordinates of each missing character within the missing region. Since the order of the candidate characters in the candidate repair text corresponds exactly to the order of the missing characters within the missing region, the filling position of the candidate character can be obtained by finding the coordinates of the missing character with the same index in the missing region location information. The semantic embedding vector of the candidate character is extracted from the ancient text-specific language model. During the generation of the candidate repair text, each candidate character in the ancient text-specific language model has already obtained a corresponding semantic embedding vector within the model. The semantic embedding vectors of each candidate character output by the model are directly read and used as the text features of that candidate character.

[0053] S51. At the fill location in the enhanced image, extract the stroke direction features and character structure features of the local image region centered on the fill location, and concatenate the stroke direction features and character structure features into the structural features of the candidate character. Specifically: In the enhanced image, the filling position is located, determined by the location information of the missing area. A local image region is defined centered on the coordinates of the filling position. The size of the local image region is preset to a fixed size, sufficient to accommodate the stroke information of a complete character. Stroke direction features are extracted from the local image region through multi-directional gradient filtering. Within the local image region, the brightness variation intensity of each pixel in multiple directions is calculated. The brightness variation intensities of all pixels in each direction are accumulated and statistically analyzed to form a feature vector describing the distribution density of strokes in different directions within the local image region. Character structure features are extracted from the local image region through spatial layout analysis. The local image region is divided into multiple sub-blocks, and the brightness distribution and texture pattern of pixels within each sub-block are statistically analyzed. These statistical results are combined to form a feature vector describing the spatial structure of the character within the local image region. The stroke direction features and character structure features are concatenated sequentially to form the structural features of the candidate character. The dimension of the concatenated structural feature vector is equal to the sum of the stroke direction feature dimension and the character shape structure feature dimension, where the first half is the stroke direction feature and the second half is the character shape structure feature.

[0054] S52. The text feature sequence of the candidate text is constructed using the text features of all candidate characters in the candidate text to be repaired, and the structural feature sequence of the candidate text is constructed using the structural features corresponding to each candidate character in the enhanced image. Specifically: Text features of all candidate characters in the candidate text to be repaired are collected. The text features of each candidate character are then arranged sequentially according to their order within the candidate text, forming a text feature sequence. The length of the text feature sequence is equal to the length of the candidate text. Structural features corresponding to each candidate character in the enhanced image are then collected. Specifically, the structural features of each candidate character are arranged sequentially according to their order within the candidate text, forming a structural feature sequence. The length of the structural feature sequence is the same as the length of the text feature sequence, and structural features at the same position in the sequence correspond to text features at the filling position.

[0055] S53. Using each text feature in the text feature sequence as the query vector and the corresponding structural feature in the structural feature sequence as the key vector, calculate the attention weight between each text feature and the corresponding structural feature. Then, perform a weighted summation of each text feature and the corresponding structural feature based on the attention weight to obtain the fusion feature of each candidate character. Specifically: For each text feature in the text feature sequence, this text feature is used as a query vector. The dimension of the query vector is equal to the dimension of the text feature. In the structural feature sequence, the structural feature at the same position as the current text feature is obtained, and this structural feature is used as the key-value vector. The dimension of the key-value vector is equal to the dimension of the structural feature. The attention weight between the query vector and the key-value vector is calculated. The attention weight is calculated by performing a dot product operation on the query vector and the key-value vector, resulting in a scalar value that reflects the correlation strength between them. The higher the calculated attention weight, the higher the degree of matching between the text semantic information of the candidate character and the local visual structure at the corresponding filling position in the enhanced image. The text feature and the corresponding structural feature are weighted and summed according to the attention weight. The text feature vector is multiplied by the attention weight, and the structural feature vector is multiplied by one minus the attention weight. The products are then added together to obtain the fused feature of the candidate character. The fused feature integrates text semantic information and image structural information, and the attention weight determines the relative contribution ratio of the two types of information during fusion.

[0056] S54. Accumulate the fusion features of all candidate characters in the candidate repair text to obtain the multimodal fusion confidence of the candidate repair text. Specifically: The fusion features of all candidate characters in the candidate repair text are obtained. Each candidate character has a corresponding fusion feature vector after processing in step S53. The fusion feature vectors of all candidate characters are accumulated according to the order of the corresponding candidate characters in the candidate repair text. The accumulation method is vector addition, adding all fusion feature vectors element by element to obtain a summary vector. The accumulated summary vector is used as the multimodal fusion confidence of the candidate repair text. The multimodal fusion confidence is represented in vector form, and each dimension of the vector reflects the cumulative response strength of the candidate repair text to the comprehensive information of text semantics and image structure across the entire text. The vector length of the multimodal fusion confidence is the same as the vector length of a single fusion feature.

[0057] In this context, candidate characters refer to the specific characters at each position in the candidate restored text generated by the ancient text-specific language model. Each candidate restored text consists of multiple candidate characters arranged sequentially, with each candidate character corresponding to a filling position in the missing region. The filling position refers to the specific spatial coordinates where the candidate character should be filled in the enhanced image. The filling position is determined by the positional information of the missing region; each missing character position in the missing region corresponds to a filling position, and each candidate character in the candidate restored text corresponds to a filling position sequentially. Semantic embedding vectors refer to the vector representation obtained by mapping text symbols to a high-dimensional continuous vector space. Semantic embedding vectors encode the semantic information and syntactic functions of the text; semantically similar texts have a shorter vector distance in high-dimensional space. When the ancient text-specific language model generates candidate characters, the candidate characters exist internally within the model in the form of semantic embedding vectors.

[0058] Among them, text features refer to the vector representations of candidate characters used for multimodal fusion. Text features directly adopt the semantic embedding vectors of candidate characters, which contain the semantic information of the candidate characters in the context and the generation confidence information given by the language model. Stroke direction features refer to feature vectors extracted from local image regions that reflect the distribution of the direction of the character strokes. Stroke direction features are obtained by calculating the directional gradient of local image regions and statistically analyzing the edge response intensity in different directions within the region to form a feature representation describing the distribution of strokes in each direction. Character structure features refer to feature vectors extracted from local image regions that reflect the shape and spatial structure of the characters. Character structure features are obtained by performing spatial texture analysis on local image regions, describing the spatial distribution pattern of pixels within the region, including structural information such as the thickness, density, and spatial extension of strokes. Structural features refer to the image feature vectors corresponding to the candidate characters used for multimodal fusion. Structural features are composed of stroke direction features and character structure features, comprehensively reflecting the visual morphological information of the local image region at the filling position. The text feature sequence refers to the ordered sequence formed by arranging the text features of all candidate characters in the candidate repair text according to the order of the candidate characters in the text. The length of the text feature sequence is equal to the total number of candidate characters in the candidate text to be repaired. The structural feature sequence refers to the ordered sequence formed by arranging the structural features corresponding to each candidate character in the enhanced image according to the order of the candidate characters in the candidate text to be repaired. Each structural feature in the structural feature sequence corresponds spatially to the text feature at the middle position of the text feature sequence.

[0059] In this context, the query vector refers to the vector representation used to initiate the matching query during attention calculation. In the multimodal fusion process, text features serve as the query vector, used to retrieve the visual information most relevant to the text semantics from structural features. The key-value vector refers to the vector representation of the queried object during attention calculation. In the multimodal fusion process, structural features serve as the key-value vector, awaiting query and matching from text features. The attention weight is a numerical weight obtained by calculating the correlation between the query vector and the key-value vector. The attention weight reflects the degree of matching between text features and structural features; a higher value indicates a higher degree of fit between the text candidate characters and the local structure of the image. The fusion feature is the comprehensive vector representation obtained by weighting and summing the text features and structural features according to the attention weights. The fusion feature encodes both text semantic information and image structural information. The multimodal fusion confidence score is the overall credibility score of the candidate repair text after comprehensively considering both text semantic and image structural information. The multimodal fusion confidence score is obtained by accumulating the fusion features of all candidate characters in the candidate repair text, used to select the optimal repair result from multiple candidate repair texts.

[0060] S6. Based on the multimodal fusion confidence of each candidate repair text, select the optimal repair text from multiple candidate repair texts, fill the missing area with the optimal repair text, and obtain the repaired text. Specifically: S60. Obtain the multimodal fusion confidence scores of each candidate repair text calculated in step S54. Each candidate repair text corresponds to a multimodal fusion confidence score vector, which comprehensively reflects the degree of matching between the candidate repair text and the corresponding positional structural features of the enhanced image across the entire text range.

[0061] S61. Compare the multimodal fusion confidence scores of each candidate restored text. The comparison method is to calculate the modulus of each multimodal fusion confidence vector, that is, the square root of the sum of the squares of the values ​​of each dimension in the vector. The calculation result is used as the comprehensive score of the candidate restored text. The higher the comprehensive score, the better the overall fit of the candidate restored text in terms of both text semantics and image structure.

[0062] S62. Sort all candidate repair texts in descending order of their comprehensive scores. The candidate repair text with the highest comprehensive score is the optimal repair text. Obtain all candidate characters from the optimal repair text. The candidate characters in the optimal repair text are arranged in order, and each candidate character corresponds to a specific filling position in the missing area. Fill each candidate character from the optimal repair text into the corresponding filling position in the missing area of ​​the enhanced image. According to the order of the candidate characters in the optimal repair text, sequentially obtain the corresponding filling position coordinates from the missing area position information of the enhanced image, and establish a mapping relationship between the text code of the candidate character and the filling position coordinates. After completing the filling of all candidate characters, organize the complete text content after filling according to the direction and order of the text line to form the repaired text. The repaired text contains the original recognizable text before and after the missing area and the filled optimal repair character, which is a complete and gap-free text sequence.

[0063] S7. Confirm the repaired text as the final repair result.

[0064] Optionally, in the above technical solution, before determining the repaired text as the final repair result, the following steps are also included: S070. Perform compliance verification on the restored text according to the ancient text usage standard database, and determine whether the restored characters in the restored text conform to the usage standard of the corresponding historical period. Specifically: S0700. Obtain the repaired text generated in step S6. The repaired text consists of the original recognizable characters and repaired characters arranged in the order of the lines of the ancient text. Extract all repaired characters from the repaired text. Repaired characters refer to all characters in the repaired text that correspond to the positions of the missing areas. By recording the positions of the candidate characters corresponding to each position in the repaired text, extract the characters at these positions sequentially. Load the ancient text character standard library. The ancient text character standard library is loaded according to the historical period of the ancient text targeted by the repair task. The content of the character standard library is different for different historical periods. For example, the Song Dynasty character standard library is loaded for Song Dynasty ancient texts, and the Ming Dynasty character standard library is loaded for Ming Dynasty ancient texts.

[0065] S0701. For each extracted repair character, perform a query and match in the ancient text character standard database. The query and match includes the following judgments: determine whether the repair character belongs to the set of commonly used characters in this historical period; if the repair character has a corresponding variant character, determine whether the form of the variant character conforms to the writing habits of this historical period; if the repair character has a corresponding phonetic loan character, determine whether its usage in the context conforms to the phonetic loan rules of this historical period.

[0066] S0702. Based on the query matching results of all repaired characters, determine whether the repaired text passes the compliance check. If all repaired characters conform to the historical usage standards in the Ancient Books Usage Standards Database, the repaired text is deemed to have passed the compliance check. If any repaired character does not conform to the historical usage standards in the Ancient Books Usage Standards Database, the repaired text is deemed to have failed the compliance check.

[0067] S071. When the repaired text passes the compliance check, it is determined as the final repair result. Specifically, when the repaired text passes the compliance check, meaning all repaired characters conform to the usage standards of the corresponding historical period in the ancient book character usage standard library, the repaired text is retained as the final repair result. Output the repaired text that has passed the compliance check, and the entire ancient book text repair process ends.

[0068] S072. When the repaired text fails the compliance check, alternative texts are selected sequentially from the remaining candidate repaired texts according to the multimodal fusion confidence score. The alternative texts are then filled into the missing areas to obtain the updated repaired text. The updated repaired text is then subjected to compliance check again until it passes the compliance check. Specifically: S0720. When the repaired text fails the compliance check, meaning at least one repaired character does not conform to the usage standards of the corresponding historical period in the ancient text usage standard library, the alternative text selection process is initiated. From the multiple candidate repair texts generated in step S4, previously selected candidate repair texts are excluded. Previously selected candidate repair texts include the earliest candidate repair text selected as the optimal repair text and candidate repair texts that have been selected as alternative texts in previous iterations. From the remaining candidate repair texts, the candidate repair text with the highest comprehensive score is selected as the alternative text, in descending order of multimodal fusion confidence.

[0069] S0721. Fill all candidate characters from the replacement text into the corresponding positions in the missing area in sequence to obtain the updated and repaired text. The filling method is the same as that used in step S6 to fill the missing area with the optimal repaired text. Return the updated and repaired text to step S070 and re-execute the compliance check. The check method is the same as that used in step S070 to check the original repaired text.

[0070] S0722. If the updated repair text passes the compliance check, proceed to step S071 to determine the updated repair text as the final repair result. If the updated repair text still fails the compliance check, repeat step S072 to select the next alternative text from the remaining candidate repair texts and fill in the missing area to obtain a new updated repair text, and then perform compliance check again. Repeat the iterative process of selecting alternative text, filling in the missing area, and performing compliance check until an updated repair text passes the compliance check. Once the check is passed, the updated repair text is determined as the final repair result.

[0071] Determining the repaired text as the final repair result includes: identifying the updated repaired text that passes compliance verification as the final repair result. Specifically, through an automatic verification and feedback mechanism, the repaired text is verified for compliance based on the ancient book character usage standard library. If it fails, alternative texts are selected sequentially according to the multimodal fusion confidence level for re-verification. This effectively reduces the reliance on large-scale, high-quality labeled data, improves the model's generalization ability and practicality in real ancient book scenarios, provides efficient and reliable technical support for large-scale ancient book digitization projects, and promotes the intelligent development of cultural heritage protection.

[0072] Optionally, the above technical solution also includes: if the candidate repair texts still fail the compliance verification after traversal, generating a manual review prompt message, and outputting all candidate repair texts and their corresponding multimodal fusion confidence scores to the user terminal, specifically: During the iteration of step S072, the system records the number of candidate repair texts selected each time a replacement text selection operation is performed. The system also records the total number of candidate repair texts. When the number of selected candidate repair texts equals the total number of candidate repair texts, the candidate repair text traversal is considered complete. At this point, all candidate repair texts have been attempted to fill the missing areas and have undergone compliance verification, but none of them have passed the compliance verification. A manual review prompt is generated, containing an explanation that automatic repair is not possible, indicating that the missing areas of the current ancient book image cannot be repaired to meet the historical usage standards determined by the system's automatic verification mechanism. All candidate repair texts and their corresponding multimodal fusion confidence scores are obtained. All candidate repair texts are those output by the ancient book-specific language model in step S4, and each candidate repair text is accompanied by a multimodal fusion confidence score calculated in step S54. The multimodal fusion confidence scores are arranged in descending order, corresponding one-to-one with the candidate repair texts. The manual review prompts, all candidate texts for repair, and their corresponding multimodal fusion confidence scores are transmitted to the user terminal via a data interface. The user terminal displays each candidate text for repair and its corresponding multimodal fusion confidence score in a list format for viewing and comparison. After viewing all candidate texts on the user terminal, the user manually selects one for repair based on their professional knowledge and judgment of ancient texts. The user terminal then sends the selected text back to the system, which fills in the missing area as the final repair result.

[0073] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation. The scheme after adjusting the order is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0074] like Figure 2 As shown in the figure, an ancient text restoration system based on generative artificial intelligence according to an embodiment of the present invention includes an image processing module, a reconstruction and extraction module, a context acquisition module, a generation module, a confidence acquisition module, a filtering and filling module, and a determination module; The image processing module is used to: perform layout analysis and preprocessing on the acquired images of ancient books to be restored, and obtain the preprocessed image; The reconstruction and extraction module is used to: perform semantic-level reconstruction of the degraded regions in the preprocessed image using a denoising diffusion probability model to obtain an enhanced image, and extract recognizable text and missing region location information from the enhanced image; The context acquisition module is used to: obtain the context text of the missing region based on the recognizable text and the location information of the missing region; The generation module is used to: input the context text into the ancient book-specific language model and generate multiple candidate repair texts; The confidence acquisition module is used to: align multiple candidate repair texts with the enhanced image, and use an attention mechanism to perform weighted fusion of the text features of the candidate repair texts and the structural features of the corresponding positions in the enhanced image to obtain the multimodal fusion confidence of each candidate repair text; The filtering and filling module is used to: select the best repair text from multiple candidate repair texts based on the multimodal fusion confidence of each candidate repair text, fill the missing area with the best repair text, and obtain the repaired text; The determination module is used to: determine the repaired text as the final repair result.

[0075] Optionally, the above technical solution also includes a compliance verification module, which is used for: Before determining the repaired text as the final repair result, the repaired text is checked for compliance according to the ancient book character usage standard library to determine whether the repaired characters in the repaired text conform to the character usage standard of the corresponding historical period. When the repaired text passes the compliance check, the repaired text will be determined as the final repair result; When the repaired text fails the compliance check, alternative texts are selected from the remaining candidate repaired texts according to the multimodal fusion confidence score. The alternative texts are then filled into the missing areas to obtain the updated repaired text. The updated repaired text is then subjected to compliance check again until the updated repaired text passes the compliance check. The module is specifically used to determine the updated repair text that has passed compliance verification as the final repair result.

[0076] Optionally, in the above technical solution, the compliance verification module is also used to: if the candidate repair text fails to pass the compliance verification after traversal, generate a manual review prompt message, and output all candidate repair texts and their corresponding multimodal fusion confidence scores to the user terminal.

[0077] Optionally, in the above technical solution, the image processing module is specifically used for: The layout of the ancient book image to be restored is analyzed to detect the text area, image area and missing area in the ancient book image to be restored, and the text area, image area and missing area are segmented and labeled to obtain the location information of the missing area; The ancient book image to be restored after segmentation and annotation is preprocessed. The preprocessing includes image denoising and contrast adaptive enhancement of the text region to obtain the preprocessed image.

[0078] Optionally, in the above technical solution, the reconstruction and extraction module is specifically used for: The trained visual Transformer network is used to extract global structural features from the preprocessed image; The global structural features are used as conditional information to input the denoising diffusion probability model. The denoising diffusion probability model is used to perform step-by-step denoising on the degraded regions in the preprocessed image. At each denoising stage, the degraded regions are repaired based on the global structural features to generate the repaired image regions. The restored image region is fused with the non-degraded region in the preprocessed image to obtain the enhanced image; Optical character recognition is performed on the enhanced image to extract the recognizable characters in the enhanced image, and the location information of the missing area is extracted from the location of the missing area in the enhanced image based on the location information of the missing area obtained from the layout analysis.

[0079] Optionally, in the above technical solution, the context acquisition module is specifically used for: Based on the location information of the missing region, determine the direction of the text line to which the missing region belongs in the enhanced image, and determine the relative position of the missing region in the text line to which it belongs; From the recognizable text, extract a preset number of recognizable characters before the missing area in the text line to which the missing area belongs as the preceding text, and extract a preset number of recognizable characters after the missing area in the text line to which the missing area belongs as the following text. The preceding and following texts are concatenated in the order of the text lines to obtain the context text of the missing area. When the number of recognizable characters before the missing area in the text line to which the missing area belongs is less than the preset number, some recognizable characters at the end of the text line above the missing area are extracted and added to the preceding text. When the number of recognizable characters after the missing area in the text line to which the missing area belongs is less than the preset number, some recognizable characters at the beginning of the text line below the missing area are extracted and added to the following text.

[0080] Optionally, in the above technical solution, the confidence level acquisition module is specifically used for: For each candidate character in the candidate repair text, the filling position of the candidate character in the enhanced image is determined based on the missing region location information, and the semantic embedding vector of the candidate character is used as the text feature of the candidate character. In the enhanced image, at the filling location, the stroke direction features and character structure features of the local image region centered on the filling location are extracted, and the stroke direction features and character structure features are concatenated to form the structural features of the candidate character. The text feature sequence of the candidate text is constructed by the text features of all candidate characters in the candidate text to be repaired, and the structural feature sequence of the candidate text is constructed by the structural features of each candidate character in the enhanced image. Using each text feature in the text feature sequence as the query vector and the corresponding structural feature in the structural feature sequence as the key vector, the attention weight between each text feature and the corresponding structural feature is calculated. Based on the attention weight, each text feature and the corresponding structural feature are weighted and summed to obtain the fusion feature of each candidate character. The multimodal fusion confidence of the candidate text is obtained by summing the fusion features of all candidate characters in the candidate repair text.

[0081] In another embodiment, the system of the present invention includes: 1) Ancient Text-Specific Language Model: This module employs a large-scale pre-trained language model optimized for Classical Chinese, capable of understanding and generating text content consistent with historical context. Based on a Transformer decoder architecture with 1.8 billion parameters, the model undergoes further pre-training on a large corpus of ancient texts (such as the *Siku Quanshu* and the *Yongle Encyclopedia*). Through a self-supervised masked language modeling task, it learns the vocabulary distribution, grammatical structure, and historical word usage patterns of Classical Chinese. Subsequently, supervised fine-tuning is performed using specialized labeled data for ancient text restoration tasks, enabling the model to accurately infer missing characters based on contextual semantics and historical word usage norms. Pre-training on a large number of ancient texts and further fine-tuning on ancient characters from specific dynasties (such as the Song, Yuan, Ming, and Qing dynasties) ensures that the language style of the generated content matches the historical period. During inference, the model receives the contextual text of the missing region, uses autoregressive text generation combined with a bundle search algorithm, and outputs multiple candidate restored texts and their generation confidence scores. This module intelligently infers and fills in missing characters based on contextual semantic understanding and Classical Chinese grammatical rules, ensuring the coherence and accuracy of the restored text. This module is specifically designed for the characteristics of ancient texts, effectively solving the problems of temporal misalignment and semantic bias that exist in general-purpose language models when processing classical Chinese.

[0082] 2) Image Recognition and Enhancement Module: This module employs cutting-edge visual Transformer networks (Vision Foundation Models) and denoising diffusion probabilistic models (Diffusion Generation Technology, Denoising Diffusion Probabilistic Models, DDPM) to perform high-precision analysis and content-aware restoration of ancient book images. First, based on the visual Transformer network architecture, the module extracts global structural features from the preprocessed image, dividing the image into multiple fixed-size image blocks. A multi-layer self-attention mechanism is then used to obtain global dependencies covering the entire image. Subsequently, the global structural features are input as conditional information into the denoising diffusion probabilistic model. This model performs a multi-step inverse denoising process starting from a purely noisy image. At each denoising stage, content-aware restoration of degraded regions is performed based on the global structural features, generating restored image regions, which are then fused with non-degraded regions to obtain the enhanced image. Simultaneously, a self-supervised pre-training strategy is introduced to learn robust degradation pattern representations on unlabeled ancient book images, effectively improving the restoration capability of low-quality images. This module performs intelligent denoising, adaptive contrast enhancement, semantic filling of damaged areas, and character structure restoration on ancient book images, outputting high-fidelity, highly readable images that provide reliable visual input for subsequent text recognition and semantic restoration. This module abandons the traditional processing method that relies solely on local convolution, integrating global modeling capabilities based on attention mechanisms with generative restoration strategies. It achieves high-quality enhancement of severely damaged ancient book images with context awareness and structural preservation, significantly outperforming traditional CNN methods in detail restoration and semantic consistency.

[0083] 3) Multimodal Fusion Inference Module: This module integrates information from the language model and image processing module, constructing a joint representation space to achieve deep interaction between text content and image features. For each candidate text to be restored generated by the ancient text-specific language model, the module obtains the semantic embedding vector of each candidate character as a text feature. Simultaneously, it extracts stroke direction features and character structure features from the corresponding filling positions in the enhanced image, concatenating them into structural features. The text features of all candidate characters are used to form a text feature sequence, and the structural features are used to form a structural feature sequence. Using the text features as the query vector and the structural features as the key vector, attention weights are calculated. The two are then weighted and summed according to the attention weights to obtain the fusion features of each candidate character. All fusion features are then summed to obtain the multimodal fusion confidence of the candidate text to be restored. In this way, the semantic and visual consistency of each candidate text is comprehensively measured. This module comprehensively considers both textual and image information, accurately inferring the content of the missing parts, ensuring that the restoration result meets both visual and semantic requirements. This module innovatively combines text understanding and image recognition, solving the problem that traditional single-dimensional methods struggle to achieve both simultaneously.

[0084] 4) Automatic Verification and Feedback Mechanism Module: This module automatically verifies the restoration results based on the unique rules of ancient texts (such as usage habits and variant characters), and provides improvement suggestions or directly corrects errors. Specifically, the module loads a library of standardized characters used in ancient texts and performs compliance verification on each restored character in the restored text, determining whether each restored character conforms to the set of commonly used characters, variant character forms, and phonetic loan rules of the corresponding historical period. If all verifications pass, the restored text is determined as the final restoration result; if not, alternative texts are selected sequentially from the remaining candidate restoration texts according to the multimodal fusion confidence score, refilled into the missing area, and verified again until verification is passed or all candidates are traversed. If all candidates fail, a manual review prompt is generated, and all candidate restoration texts and their corresponding multimodal fusion confidence scores are output to the user for manual selection. This module verifies the historical accuracy of the restoration results, reduces the need for human intervention, and improves work efficiency. This module implements an automated quality control process, improving the overall system reliability and usability.

[0085] The system workflow in this embodiment is as follows: 1) Input Stage: The user uploads the image of the ancient book to be restored to the system. The image recognition and enhancement module first performs the layout analysis and preprocessing operation in step S1. Specifically, ① the layout analysis of the ancient book image to be restored is performed to detect the text regions, image regions, and missing regions, and each region is segmented and labeled to obtain the location information of the missing regions. ② Image denoising and contrast adaptive enhancement processing are performed on the segmented and labeled text regions to obtain the preprocessed image. Subsequently, the image recognition and enhancement module performs the image enhancement operation in step S2. Specifically, ① a trained visual Transformer network is used to extract the global structural features of the preprocessed image. ② The global structural features are used as conditional information input to the denoising diffusion probability model. The denoising diffusion probability model performs step-by-step denoising processing on the degraded regions in the preprocessed image, and content-aware restoration is performed on the degraded regions based on the global structural features at each denoising stage to generate the restored image region. ③ The restored image region is fused with the non-degraded regions in the preprocessed image to obtain the enhanced image. Through the above processing, the accuracy of optical character recognition in subsequent steps is improved.

[0086] 2) Text Restoration Stage: The ancient text-specific language model infers possible text content based on contextual information, while the enhanced image provided by the image recognition and enhancement module serves as a supplement, jointly guiding the restoration process. In the first aspect, the image recognition and enhancement module performs optical character recognition on the enhanced image, extracting recognizable characters. Based on the location information of the missing area obtained from the layout analysis, it extracts the location information of the missing area from the location of the missing area in the enhanced image. Subsequently, based on the recognizable characters and the location information of the missing area, the contextual text of the missing area is obtained: determining the direction of the text line to which the missing area belongs in the enhanced image and its relative position within that text line; extracting a preset number of recognizable characters from the text line to which the missing area belongs as the preceding text, and extracting a preset number of recognizable characters from the text line to which the missing area belongs as the following text; concatenating the preceding and following texts according to the text line direction; when the number of recognizable characters is less than the preset number, extracting additional characters from the previous or next text line to obtain the contextual text of the missing area. Secondly, the contextual text is input into a language model specifically designed for ancient texts. This model generates multiple candidate texts for restoration and their generation confidence scores through autoregressive text generation and bundle search algorithms. The text restoration stage integrates the information from both aspects, providing a set of candidate texts and image features for subsequent fusion inference.

[0087] 3) Fusion Inference Stage: The multimodal fusion inference module integrates the information from the above two aspects to generate the final text restoration suggestions. This module aligns multiple candidate restoration texts with the enhanced image, and uses an attention mechanism to weightedly fuse the text features of the candidate restoration texts with the structural features of the corresponding positions in the enhanced image to obtain the multimodal fusion confidence of each candidate restoration text. Specifically, for each candidate character in each candidate restoration text, the filling position of the candidate character in the enhanced image is determined based on the missing region location information, and the semantic embedding vector of the candidate character is used as the text feature; the stroke direction features and character structure features of the local image region at the filling position in the enhanced image are extracted and concatenated into structural features; a text feature sequence is constructed using the text features of all candidate characters, and a structural feature sequence is constructed using the corresponding structural features; each text feature in the text feature sequence is used as a query vector, and the structural features at the corresponding positions in the structural feature sequence are used as key vectors to calculate attention weights. The text features and structural features are weighted and summed according to the attention weights to obtain the fusion features of each candidate character; the fusion features of all candidate characters are accumulated to obtain the multimodal fusion confidence of the candidate restoration text. Finally, based on the multimodal fusion confidence of each candidate repair text, the optimal repair text is selected from multiple candidate repair texts and filled into the missing region to obtain the repaired text. Through the above fusion reasoning, it is ensured that the repair result meets both visual and semantic requirements.

[0088] 4) Verification and Feedback Phase: The repair results are evaluated and adjusted as necessary until a satisfactory repair effect is achieved. This mechanism verifies the compliance of the repaired text based on the ancient text usage standard library, determining whether the repaired characters in the repaired text conform to the usage standards of the corresponding historical period. When the repaired text passes the compliance verification, it is determined as the final repair result. When the repaired text fails the compliance verification, replacement texts are sequentially selected from the remaining candidate repair texts according to the multimodal fusion confidence score. The replacement texts are then used to fill the missing areas to obtain the updated repaired text, and the updated repaired text is verified for compliance again until it passes the compliance verification. If the candidate repair texts still fail the compliance verification after traversing all the candidate repair texts, a manual review prompt is generated, and all candidate repair texts and their corresponding multimodal fusion confidence scores are output to the user's end, allowing the user to manually select a repair text based on their professional knowledge. Through the above automated verification and iterative adjustment process, the historical accuracy of the repair results is verified, the need for human intervention is reduced, and work efficiency is improved.

[0089] To more clearly demonstrate the technical solution of this invention, the following specific embodiments illustrate how to apply the ancient book text restoration method based on generative artificial intelligence to the ancient book digital preservation system, especially in the specific implementation in cultural heritage institutions, digital humanities platforms, and ancient book collation software. This embodiment adopts an architecture that combines a language model specifically for ancient books with image enhancement technology based on visual Transformer networks and denoising diffusion probability models, enabling high-precision and semantically consistent text restoration of damaged ancient book images.

[0090] Example 1: Intelligent Restoration Platform for Ancient Books: This example describes an intelligent restoration platform for ancient books for libraries and document preservation institutions. It integrates a language model specifically for ancient books, image recognition and enhancement modules, and an automatic verification and feedback mechanism to achieve an end-to-end restoration process from image input to text output. The system architecture is as follows: 1) The Ancient Texts-Specific Language Model Module is based on a large-scale language model that has undergone pre-training and supervised fine-tuning on a massive corpus of ancient texts (such as the *Siku Quanshu* and the *Yongle Encyclopedia*), enabling it to understand and generate classical Chinese texts. This model employs a Transformer decoder architecture with 1.8 billion parameters. During the pre-training phase, a self-supervised masked language modeling task allows the model to learn the vocabulary distribution, grammatical structure, and historical word usage patterns of classical Chinese. The supervised fine-tuning phase uses specialized labeled data from ancient text restoration tasks, enabling the model to accurately infer missing characters based on contextual semantics and historical word usage norms. When the system receives contextual fragments (such as preceding and following complete sentences or adjacent characters), the model generates multiple candidate restoration texts based on historical word usage habits, grammatical structure, and semantic logic, using autoregressive text generation and bundle search algorithms, and outputs them sorted by generation confidence scores.

[0091] 2) The image recognition and enhancement module uses a trained visual Transformer network to extract global structural features from ancient book images and combines this with a denoising diffusion probability model to perform semantic-level reconstruction of degraded areas such as insect damage, ink stains, and paper aging. The visual Transformer network divides the preprocessed image into multiple fixed-size image blocks and obtains global structural features covering the entire image through a multi-layer self-attention mechanism. The denoising diffusion probability model performs a multi-step inverse denoising process starting from a purely noisy image. At each denoising stage, it performs content-aware restoration of degraded areas based on global structural features, generating restored image regions, which are then fused with non-degraded regions in the preprocessed image to output the enhanced image. This module also provides more reliable character location information for the optical character recognition (OCR engine).

[0092] 3) The multimodal fusion inference module aligns the candidate restoration text generated by the ancient text-specific language model with the character position and stroke structure information provided by the image recognition and enhancement module. Through a weighted fusion using an attention mechanism, it selects the restoration result that best matches the visual form and semantic logic. Specifically, for each candidate character in each candidate restoration text, the filling position of the candidate character in the enhanced image is determined based on the missing region location information. The semantic embedding vector of the candidate character is used as the text feature. Stroke direction features and character structure features are extracted at the filling position in the enhanced image and concatenated to form structural features. A text feature sequence is constructed using the text features of all candidate characters, and a structural feature sequence is constructed using the corresponding structural features. Each text feature in the text feature sequence is used as a query vector, and the structural features at the corresponding position in the structural feature sequence are used as key vectors. Attention weights are calculated, and the text features and structural features are weighted and summed according to the attention weights to obtain the fusion features of each candidate character. The fusion features of all candidate characters are accumulated to obtain the multimodal fusion confidence of the candidate restoration text. Finally, the optimal restoration text is selected based on the multimodal fusion confidence scores.

[0093] 4) The automatic verification and feedback mechanism module uses a database of ancient text usage standards (such as a list of variant characters, mapping of phonetic loan characters, and frequency statistics of characters from different eras) to verify the compliance of the repair results. It determines whether the repaired characters in the restored text conform to the usage standards of the corresponding historical period. If a misalignment in era or semantic conflict is found, the system automatically selects replacement text from the remaining candidate repair texts according to the multimodal fusion confidence level, re-fills the missing area, and verifies it again, or prompts for manual review.

[0094] The operation process is as follows: 1) Users upload scanned images of ancient books to be restored to the platform, and the system automatically performs layout analysis and image preprocessing in step S1. Layout analysis detects and segments text regions, image regions, and missing regions, and obtains the location information of missing regions; preprocessing performs image denoising and adaptive contrast enhancement on text regions to obtain the preprocessed image.

[0095] 2) The image recognition and enhancement module enhances the image by performing semantic-level reconstruction using the denoising diffusion probability model in step S2. A trained visual Transformer network is used to extract global structural features from the preprocessed image. These global structural features are then input into the denoising diffusion probability model as conditional information. Content-aware restoration of degraded regions is performed through step-by-step denoising, generating restored image regions that are then fused with non-degraded regions to output the enhanced image.

[0096] 3) The optical character recognition engine extracts recognizable text, executes step S3 to obtain the context text of the missing area, and sends the context text into the ancient book-specific language model to generate candidate repair text.

[0097] 4) The multimodal fusion inference module integrates the enhanced image and candidate repair text information, executes steps S5 and S6 to determine the optimal repair text and fills the missing area to obtain the repaired text; the automatic verification and feedback mechanism executes steps S070 to S072 to verify its historical compliance and outputs the final repair result.

[0098] Example 2: Integration into a University Digital Humanities Research Platform. This example demonstrates how to integrate the ancient text restoration method of this invention into a university digital humanities research platform, serving academic scenarios such as ancient text collation, textual criticism, and historical linguistic analysis. System Architecture: 1) The platform deploys a domain-fine-tuned language model specifically designed for ancient texts, optimized for specific historical periods (such as the Tang and Song dynasties, and the Ming and Qing dynasties) or document types (such as local gazetteers, medical books, and legal codes). This model employs a Transformer decoder architecture, undergoes further pre-training on a large-scale corpus of ancient texts, and is subject to differentiated supervised fine-tuning based on the usage norms of different historical periods and document type characteristics. Researchers can specify the historical context or document category of the restoration task, and the model adjusts its usage preferences and semantic reasoning strategies accordingly to improve the historical accuracy of the restoration results.

[0099] 2) Image Recognition and Enhancement Module: This module utilizes an image processing method based on Visual Transformer and a denoising diffusion probability model to perform refined restoration of high-resolution scanned ancient books. Visual Transformer extracts global structural features from the preprocessed image, while the denoising diffusion probability model performs content-aware restoration of degraded areas based on these global structural features during the inverse denoising process. The restored image region is then fused with the non-degraded region to obtain the enhanced image. This module supports batch processing of entire pages of images and automatically labels areas with low confidence for researchers to focus on reviewing.

[0100] 3) During the restoration process, the multimodal fusion inference module synchronously compares the remaining stroke directions in the enhanced image with the candidate characters generated by the ancient text-specific language model. It achieves consistency among character structure, writing style, and contextual semantics, avoiding issues such as "semantic correct but character shape mismatch" or "similar character shape but misaligned era." Specifically, for each candidate character in each candidate text to be restored, stroke direction features and character structure features are extracted from the corresponding filling position in the enhanced image as structural features. The semantic embedding vector of the candidate character is used as text features. Attention weights are calculated through an attention mechanism, and the text features and structural features are weighted and summed to obtain fusion features. The optimal restored text is then selected based on these fusion features.

[0101] 4) The platform provides a multi-person collaborative proofreading function, allowing multiple researchers to annotate, discuss, or revise the same restoration result. It also supports automatic comparison of the restored text with known extant versions to help determine the source of variant texts or copying errors.

[0102] The operation process is as follows: 1) Researchers upload the image collection of ancient books to be organized (such as a full scan of an unpublished manuscript) and select the corresponding document type and historical period label.

[0103] 2) The system automatically performs image enhancement and preliminary optical character recognition in steps S1 and S2, marking low-confidence or missing regions. The image recognition and enhancement module performs layout analysis, preprocessing, and semantic-level reconstruction using a noise reduction diffusion probability model, outputting the enhanced image; the optical character recognition module extracts recognizable text and the location information of missing regions from the enhanced image.

[0104] 3) The ancient book-specific language model generates candidate repair texts by combining the context. The multimodal fusion reasoning module fuses the visual features of the enhanced image with the text features of the candidate repair texts. Steps S5 and S6 are executed to output the optimal repair text and fill in the missing area.

[0105] 4) The repair results are presented in editable text format. Researchers can directly annotate, accept or modify them within the platform, and export them to standard formats such as TEI-XML for subsequent research.

[0106] This embodiment fully demonstrates the practicality and scalability of the invention in a professional academic environment. By closely integrating the linguistic characteristics and visual features of ancient books, the system not only improves restoration efficiency but also provides a high-quality data foundation for in-depth text analysis and version collation, effectively supporting the intelligent transformation of digital humanities research.

[0107] This invention significantly improves the intelligence and quality of ancient text restoration by combining a language model specifically optimized for ancient texts with image enhancement techniques based on a visual Transformer network and a denoising diffusion probability model. In image processing, a visual Transformer network extracts global structural features from the preprocessed image. These global structural features are then used as conditional information input to the denoising diffusion probability model. Content-aware restoration of degraded areas is performed through step-by-step denoising, generating a restored image region which is then fused with the non-degraded region to obtain an enhanced image. This effectively overcomes the shortcomings of traditional methods that rely solely on local convolution processing in terms of detail recovery and semantic consistency. In text restoration, the language model specifically for ancient texts employs a Transformer decoder architecture. It undergoes further pre-training on a large-scale corpus of ancient texts, learning the vocabulary distribution, grammatical structure, and historical word usage patterns of classical Chinese through a self-supervised masked language modeling task. Supervised fine-tuning is then performed using specialized labeled data for ancient text restoration tasks, enabling the model to accurately infer missing characters based on contextual semantics and historical word usage norms. By deeply integrating a language model specifically designed for classical Chinese contexts with a multimodal fusion reasoning module, the system can accurately infer and restore the original text that conforms to historical word usage norms and contextual semantics even when the text is severely damaged, the ink is blurred, or characters are missing. This effectively overcomes the shortcomings of traditional methods in terms of semantic coherence and time-appropriateness.

[0108] This invention, while ensuring high restoration accuracy, employs a strategy of continued pre-training and supervised fine-tuning, significantly reducing reliance on high-quality labeled data and computational resources. The ancient text-specific language model uses a base model with 1.8 billion parameters. This parameter count, under the low-resource conditions of scarce high-quality labeled data in the ancient text field, achieves a balance between model expressive power and generalization ability, effectively addressing the characteristics of low-resource and knowledge-rich ancient Chinese information processing. By performing continued pre-training on a large-scale unlabeled ancient text corpus, the model can learn the basic linguistic rules of ancient texts without manual annotation; only a small amount of specialized labeled data is needed for supervised fine-tuning to enable the model to possess accurate text restoration capabilities. This technical solution allows the system to run efficiently on ordinary servers and even edge devices, suitable for the practical deployment needs of large-scale ancient text digitization projects. Compared with traditional manual proofreading or single optical character recognition restoration methods, this invention effectively solves core problems such as low efficiency, poor generalization, and semantic distortion, significantly improving the automation and intelligence level of ancient text restoration.

[0109] Furthermore, this invention possesses a context-aware, automatic verification, and feedback mechanism, capable of dynamically optimizing the restoration results based on the linguistic patterns, character usage habits, and image structure information of ancient texts. The automatic verification and feedback mechanism loads a database of standardized characters used in ancient texts, performing compliance checks on each restored character in the restored text. It determines whether each restored character conforms to the set of commonly used characters, variant forms, and phonetic loan rules of the corresponding historical period. If a character fails the check, a replacement text is selected sequentially from the remaining candidate restored texts based on multimodal fusion confidence and re-verified. Through this mechanism, the system ensures that the restored content conforms to both visual form and historical context, thereby improving the accuracy and usability of ancient text digitization while preserving the original appearance of the documents to the greatest extent possible, and optimizing the technical approach and resource utilization efficiency for cultural heritage protection.

[0110] This invention is applicable to libraries, museums, cultural heritage protection institutions, and other places that require the digital preservation and restoration of ancient books. It can also be used for academic research and for personal enthusiasts to explore ancient text resources. Through the technical solution of this invention, not only can precious cultural heritage be effectively protected, but the widespread dissemination and development of outstanding traditional Chinese culture can also be promoted.

[0111] It should be noted that the beneficial effects of the ancient text restoration system based on generative artificial intelligence provided in the above embodiments are the same as those of the ancient text restoration method based on generative artificial intelligence described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0112] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for restoring ancient texts based on generative artificial intelligence.

[0113] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for restoring ancient texts based on generative artificial intelligence.

[0114] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0115] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for restoring ancient texts based on generative artificial intelligence, characterized in that, include: The obtained images of ancient books to be restored are subjected to layout analysis and preprocessing to obtain preprocessed images; The degraded regions in the preprocessed image are semantically reconstructed using a denoising diffusion probability model to obtain an enhanced image, and identifiable text and missing region location information are extracted from the enhanced image. Based on the recognizable text and the location information of the missing region, obtain the context text of the missing region; The context text is input into a language model specifically for ancient books to generate multiple candidate texts for repair. Align the multiple candidate repair texts with the enhanced image, and use an attention mechanism to perform weighted fusion of the text features of the candidate repair texts with the structural features of the corresponding positions in the enhanced image to obtain the multimodal fusion confidence of each candidate repair text. Based on the multimodal fusion confidence of each candidate repair text, the optimal repair text is selected from the multiple candidate repair texts, and the optimal repair text is filled into the missing region to obtain the repaired text; The repaired text is determined as the final repair result.

2. The method for restoring ancient texts based on generative artificial intelligence according to claim 1, characterized in that, Before determining the repaired text as the final repair result, the following steps are also included: The restored text is verified for compliance based on the ancient text character usage standard library to determine whether the restored characters in the restored text conform to the character usage standard of the corresponding historical period. When the repaired text passes the compliance verification, the repaired text will be determined as the final repair result; When the repaired text fails the compliance check, alternative texts are selected sequentially from the remaining candidate repaired texts according to the multimodal fusion confidence score. The alternative texts are then filled into the missing area to obtain the updated repaired text. The updated repaired text is then subjected to compliance check again until the updated repaired text passes the compliance check. Determining the repaired text as the final repair result includes: determining the updated repaired text that has passed compliance verification as the final repair result.

3. The method for restoring ancient texts based on generative artificial intelligence according to claim 2, characterized in that, Also includes: If the candidate repair text fails to pass the compliance check after being traversed, a manual review prompt message is generated, and all candidate repair texts and their corresponding multimodal fusion confidence scores are output to the user terminal.

4. The method for restoring ancient texts based on generative artificial intelligence according to claim 1, characterized in that, The obtained images of the ancient books to be restored were subjected to layout analysis and preprocessing to obtain preprocessed images, including: The layout of the ancient book image to be restored is analyzed to detect the text region, image region and missing region in the ancient book image to be restored, and the text region, the image region and the missing region are segmented and labeled to obtain the location information of the missing region; The image of the ancient book to be restored after segmentation and annotation is preprocessed, including image denoising and contrast adaptive enhancement of the text region, to obtain the preprocessed image.

5. A system for restoring ancient texts based on generative artificial intelligence, characterized in that, It includes an image processing module, a reconstruction and extraction module, a context acquisition module, a generation module, a confidence acquisition module, a filtering and input module, and a confirmation module; The image processing module is used to: perform layout analysis and preprocessing on the acquired images of ancient books to be restored, and obtain preprocessed images; The reconstruction and extraction module is used to: perform semantic-level reconstruction of the degraded region in the preprocessed image using a denoising diffusion probability model to obtain an enhanced image, and extract identifiable text and missing region location information from the enhanced image; The context acquisition module is used to: acquire the context text of the missing region based on the recognizable text and the location information of the missing region; The generation module is used to: input the context text into a language model specifically for ancient books to generate multiple candidate repair texts; The confidence acquisition module is used to: align the multiple candidate repair texts with the enhanced image, and use an attention mechanism to perform weighted fusion of the text features of the candidate repair texts with the structural features of the corresponding positions in the enhanced image to obtain the multimodal fusion confidence of each candidate repair text; The filtering and filling module is used to: filter out the optimal repair text from the multiple candidate repair texts based on the multimodal fusion confidence of each candidate repair text, and fill the optimal repair text into the missing area to obtain the repaired text; The determining module is used to: determine the repaired text as the final repair result.

6. The ancient book text restoration system based on generative artificial intelligence according to claim 5, characterized in that, It also includes a compliance verification module, which is used for: Before determining the repaired text as the final repair result, the repaired text is checked for compliance according to the ancient book character usage standard library to determine whether the repaired characters in the repaired text conform to the character usage standard of the corresponding historical period. When the repaired text passes the compliance verification, the repaired text will be determined as the final repair result; When the repaired text fails the compliance check, alternative texts are selected sequentially from the remaining candidate repaired texts according to the multimodal fusion confidence score. The alternative texts are then filled into the missing area to obtain the updated repaired text. The updated repaired text is then subjected to compliance check again until the updated repaired text passes the compliance check. The determining module is specifically used to: determine the updated repair text that has passed the compliance verification as the final repair result.

7. The ancient book text restoration system based on generative artificial intelligence according to claim 6, characterized in that, The compliance verification module is also used to: if the candidate repair text fails to pass the compliance verification after traversal, generate a manual review prompt message, and output all candidate repair texts and their corresponding multimodal fusion confidence scores to the user terminal.

8. A system for restoring ancient texts based on generative artificial intelligence according to any one of claims 5 to 7, characterized in that, The image processing module is specifically used for: The layout of the ancient book image to be restored is analyzed to detect the text region, image region and missing region in the ancient book image to be restored, and the text region, the image region and the missing region are segmented and labeled to obtain the location information of the missing region; The image of the ancient book to be restored after segmentation and annotation is preprocessed, including image denoising and contrast adaptive enhancement of the text region, to obtain the preprocessed image.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the ancient text restoration method based on generative artificial intelligence as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the ancient text restoration method based on generative artificial intelligence as described in any one of claims 1 to 4.

Images