A method and related equipment for restoring length-scale images of ancient Chinese books

Abstract

This invention discloses a method and related equipment for restoring length-scale images of ancient Chinese books, belonging to the field of ancient book restoration. The method includes: acquiring ancient book images and filtering out intact and damaged images; for intact images, simulating damage using random masking to construct training data pairs; for damaged images, marking the location of missing text and determining its specific content; constructing a restoration model based on a diffusion model and training the model; and using the trained model, gradually restoring length-scale images of ancient books through an autoregressive approach in the form of overlapping sliding windows. This invention can automatically restore length-scale damaged ancient book images based on the locations and content of damaged text marked by human experts, employing diffusion model technology combined with an autoregressive restoration method, while effectively maintaining the overall consistency of font style and background during the restoration process.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and ancient book restoration, and in particular to a method and related equipment for restoring length-scale images of ancient Chinese books. Background Technology

[0002] Chinese culture is ancient and profound, leaving behind numerous precious ancient books and documents. The texts in these documents not only contain rich historical information but also serve as important bridges for cultural transmission and inheritance. In today's information age, the widespread application of digital technology has greatly improved the efficiency of document processing, storage, analysis, and dissemination. In response to the national strategic needs for the protection of historical and cultural heritage and the digitization and promotion of ancient books, it is imperative to carry out the restoration of ancient books and documents.

[0003] In recent years, computer science and artificial intelligence methods and paradigms have played an important role in digital humanities research, promoting the development of ancient book digitization, digital heritage preservation, and information discovery. Although these technologies have achieved remarkable results, they still face many problems and challenges in processing damaged images of Chinese ancient books. (1) Existing methods mainly focus on repairing individual Chinese characters in ancient books and directly pasting the repaired character images back into the original image. This approach can easily lead to inconsistencies between the repaired area and the background of the original image. (2) Existing methods can usually only repair small image blocks (such as 512×512 pixels), which is difficult to meet the needs of repairing large-area damage or entire pages or volumes of ancient book images. Summary of the Invention

[0004] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to provide an autoregressive method and related equipment for restoring Chinese ancient book images based on a diffusion model.

[0005] The first technical solution adopted in this invention is: A method for restoring length-based images of ancient Chinese books includes the following steps: Data preparation: Acquire images of ancient books, and filter out intact and damaged images of ancient books; Data preprocessing: For intact ancient book images, damage is simulated by random masking to construct training data pairs; for damaged ancient book images, the location of missing text is marked and the specific content of the missing text is determined. Incomplete text generation: Construct a restoration model based on a diffusion model, and train the restoration model using the constructed training data to enable the model to learn the writing styles of different ancient books; Autoregressive length-level restoration: Using a trained model, high-quality restoration of ancient book images is achieved step by step through an autoregressive approach using overlapping sliding windows.

[0006] Furthermore, the data preprocessing includes: For intact ancient book images, the ancient book images are cut into image blocks of a preset size, and the text areas of the image blocks are randomly destroyed by random masking to form "intact-damaged" data pairs; For damaged images of ancient books, the locations of missing text are precisely marked using character boxes, and the marking results are recorded in a standardized format. ); For the marked locations of missing characters, the content of each missing character is restored to obtain an image with the missing characters repaired.

[0007] Furthermore, the restoration model takes the damaged image, the mask image of the damaged location, and the image of the damaged character content as input to generate a restoration result of the ancient book image that is highly consistent in character style and background; The repair model includes a text location and content decoder and a U-shaped backbone network; The text location and content decoder is used to generate a corresponding mask image of the damaged text location and a damaged text content image based on the marked damaged text location and content. The input of the U-shaped backbone network consists of the original damaged image, the damaged text content image, the damaged text location mask image, and a noise image, and outputs a restored image to achieve high-quality restoration of ancient book pages.

[0008] Furthermore, a character-supervised mechanism is used to train the repair model, with the loss function being:

[0009] In the formula, Indicates VGG network, To repair the image, For the target image, To repair the region mask image.

[0010] Furthermore, the autoregressive length-level repair includes: Obtain the image of the ancient book to be restored, set the size and sliding step of the image block, initialize the set of character boxes to be restored, and record the area where each character to be restored is located; Enter the autoregressive loop to determine if there are still any unrepaired characters; if all characters have been repaired, terminate the loop; if there are still unrepaired characters, continue with the subsequent steps. Calculate the minimum and maximum x and y coordinates of all unrepaired character boxes to form the minimum bounding rectangle of the unrepaired character boxes; use the four corners of this bounding rectangle as the starting points of candidate image blocks, and calculate the number of unrepaired character boxes completely covered by the image block corresponding to each starting point. Determine the starting point, move the window across the image according to the set sliding step size and direction, and generate image blocks in sequence; mark a character box as repaired whenever it is completely covered by an image block; if the character box is only partially covered, it is not marked as repaired. For each generated image block, create a damaged text content image and a damaged text location mask image respectively; Connectivity analysis and morphological processing are performed on the mask image of the damaged text locations; Save each image block, the damaged text content image, the damaged text location mask image and its location information to the repaired image block set until all character boxes are completely repaired and covered. For each generated image block, the size of its damaged text content image and damaged text position mask image is uniformly adjusted to the preset standard size to adapt to the input requirements of the subsequent repair model; The image patch, the image of the damaged text content, and the mask image of the location of the damaged text are input into the trained restoration model for inference to obtain the restored image patch. Paste the restored image blocks back into the original image according to their actual positions within the entire ancient book image.

[0011] Furthermore, determining the starting point includes: Select the candidate image patch with the fewest unrepaired character boxes as the starting point of the sliding window.

[0012] Furthermore, the connected component analysis and morphological processing of the mask image of the damaged text locations includes: In the horizontal and vertical directions, mask regions with intervals lower than a preset threshold are connected, and a binary filling algorithm is used to automatically fill in small holes in the mask. The area of ​​newly generated connected regions is calculated, and only small connected regions with an area lower than the minimum character box area threshold are retained and incorporated into the mask area to ensure that the broken background between character boxes can be correctly repaired.

[0013] Furthermore, before inputting the image patch, the damaged text content image, and the damaged text location mask image into the restoration model, the following steps are also included: The resized image blocks, the mask image of the damaged text location, and the damaged text content image are converted into Tensor format and then normalized.

[0014] The second technical solution adopted in this invention is: An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a length-based Chinese ancient book image restoration method as described above.

[0015] The third technical solution adopted in this invention is: A computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a length-based Chinese ancient book image restoration method as described above.

[0016] The fourth technical solution adopted in this invention is: A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method for restoring length-scale images of ancient Chinese books.

[0017] The beneficial effects of this invention are: based on the location and content of the damaged text in ancient books annotated by human experts, this invention uses diffusion model technology combined with an autoregressive restoration method to automatically restore images of damaged ancient books at the page level, and effectively maintains the overall consistency of font style and background during the restoration process. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a network architecture diagram of the repair model based on the diffusion model in this embodiment of the invention; Figure 2 This is a schematic diagram of the text position and content decoder in an embodiment of the present invention; Figure 3 This is a schematic diagram of the self-regressive length-level repair process in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the steps of a length-based Chinese ancient book image restoration method in an embodiment of the present invention. Detailed Implementation

[0020] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0021] The terminology used in the embodiments of this application is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of this application. The singular forms "a," "described," and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise expressly limited, terms such as "set," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0022] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0023] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0024] In the description of this application, "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0025] Terminology Explanation: VGG: an abbreviation for Visual Geometry Group, is a classic deep convolutional neural network (CNN) architecture.

[0026] To address existing technical problems, this invention proposes an autoregressive automatic restoration scheme for length-scale Chinese ancient book images based on a diffusion model. This scheme can efficiently restore length-scale Chinese ancient book images, effectively improving the restoration efficiency and accuracy in the digitization process of ancient books. The scheme encompasses four main stages: data preparation, data preprocessing, ancient book image block restoration, and overall length-scale restoration. In the data preparation and preprocessing stage, images of ancient books with damage characteristics are first collected from the internet, and the damaged areas and their content are precisely labeled manually to ensure a high-quality data foundation for subsequent restoration tasks. In the ancient book image block restoration stage, the entire page of the ancient book image is cropped into appropriately sized image blocks according to the processing capacity of the diffusion model, and combined with the manually labeled damage locations and content, targeted restoration of each image block is achieved. Addressing the challenge of length-scale restoration, this invention innovatively designs an autoregressive image block stitching method. Through a sliding window strategy, it performs continuous and consistent restoration of the entire page of the ancient book image, significantly improving the effect and efficiency of large-scale restoration. Compared with traditional restoration methods that mainly rely on manual labor, the restoration system proposed in this invention not only significantly shortens the restoration cycle and reduces the input of manpower and material resources, but also ensures a high degree of consistency in style and content of the restoration results, providing strong technical support for the protection and digitization of ancient Chinese books.

[0027] Example 1 like Figure 4 As shown, this embodiment provides a length-based method for restoring images of ancient Chinese books, which can not only effectively restore the overall appearance of the ancient book image, but also maintain the consistency of the restored text with the original text's writing style. The method specifically includes the following steps: S1. Data preparation: Obtain images of ancient books and filter out intact and damaged images of ancient books.

[0028] For example, images of ancient books are collected from the internet, and intact and damaged images are manually selected. In some embodiments, if the original format of the collected electronic files of ancient books is .pdf, in order to improve the efficiency of subsequent data processing, viewing and storage, special tools are used to extract images from the .pdf files and convert them into .png image files.

[0029] S2. Data Preprocessing: For intact ancient book images, damage is simulated by random masking to construct training data pairs; for damaged ancient book images, the location of missing text is marked and the specific content of the missing text is determined.

[0030] In some embodiments, step S2 specifically includes the following steps: S21. For intact images, this embodiment divides them into 512×512 pixel image blocks, and randomly damages the text areas using random masking, thus forming "intact-damaged" data pairs. Specifically, based on the position of the character frame, the character area is randomly masked at a ratio of 0.05-0.2. The masking methods include full white masking, full black masking, and blurred masking, thereby simulating various damage types of ancient documents and providing rich data support for subsequent model training.

[0031] S22. For damaged images. In this embodiment, human experts use annotation boxes to precisely mark the location of the damaged text, and the annotation results are recorded in a uniform format. The recording format is as follows: ).

[0032] S23. For the damaged character positions marked in step S22, human experts are further required to consult relevant literature to verify and restore the content of each damaged character. Finally, the manually marked damaged text positions and their corresponding content are obtained.

[0033] S3. Incomplete text generation: Construct a restoration model based on a diffusion model, and use the constructed training data to train the restoration model so that the model learns the writing styles of different ancient books.

[0034] In some embodiments, a generative network architecture based on a diffusion model is designed, which integrates text location and content decoders, a U-shaped backbone network, and a character-level supervision mechanism. This design enables the generation of ancient book image restoration results with high consistency in character style and background, taking damaged images, damaged location mask images, and damaged character content images as input. The specific network structure is as follows... Figure 1 As shown.

[0035] 1) Text location and content decoder The text location and content decoder generates corresponding mask images of the damaged text locations and images of the damaged text based on the labeled locations and contents. This process provides accurate spatial and semantic information for subsequent character generation, as illustrated in the diagram. Figure 2 As shown.

[0036] 2) U-shaped backbone network As one optional implementation, the structure diagram of the U-shaped backbone network is shown in Table 1. The image dimensions use... express.

[0037] Table 1. U-shaped backbone network structure

[0038] The input to the U-shaped backbone network consists of the original image of the damaged ancient book, an image of the damaged text content, a mask image of the damaged text location, and a noise image. The original damaged image and the noise image are 3-channel images, while the damaged text content image and the mask image of the damaged text location are single-channel images. All images are stacked along the channel dimension to form an 8-channel input vector. This 8-channel vector is fed into the U-shaped backbone network for denoising processing, ultimately outputting a 3-channel restored image, achieving high-quality restoration of the ancient book pages.

[0039] 3) Model Training To further improve the accuracy and stylistic consistency of the restored character content, this embodiment introduces a character supervision mechanism. Specifically, the character supervision mechanism utilizes a pre-trained VGG network to extract feature representations of the restored region in both the restored and target images, and constrains the model by calculating the feature differences between the two images in the restored character region. The calculation formula is as follows:

[0040] In practice, the repaired image and the target image are input into the VGG network to extract features. Then, using the mask map of the repaired region, only the features within the mask area are calculated, thereby achieving efficient supervision of the characters in the repaired region.

[0041] As one specific implementation method, the training parameters are as follows: a) Iteration Algebra: 150 b) Image resolution: 512×512 c) Optimizer: AdamW d) Learning rate: 0.0001 e) Weight decay: 0.00001.

[0042] S4. Autoregressive length-level restoration: Using the trained model, length-level restoration of ancient book images is achieved step by step in the form of overlapping sliding windows through an autoregressive approach.

[0043] In some embodiments, see Figure 3 Step S4 specifically includes the following steps: S41. Obtain the size information of the ancient book image to be restored, set the size and sliding step of the image block, initialize the set of character boxes to be restored, and record the area where each character to be restored is located.

[0044] S42. Enter the autoregressive loop, first determining whether there are still unrepaired characters. If all characters have been repaired, terminate the loop; if unrepaired characters remain, continue with the subsequent steps.

[0045] S43. Calculate the minimum and maximum x and y coordinates of all unrepaired character frames to form the minimum bounding rectangle of the unrepaired character frames. Using the top left, top right, bottom left, and bottom right corners of this bounding rectangle as the starting points of candidate image patches, calculate the number of unrepaired character frames that the image patch corresponding to each starting point can completely cover.

[0046] S44. In order to ensure that the initial image block has enough complete Chinese characters for the model to use as a reference, this embodiment selects the candidate image block with the fewest unrepaired character boxes as the starting point of the sliding window.

[0047] S45. Starting from a selected point, move the window across the image according to the set sliding step size and direction, generating image blocks sequentially. Whenever a character box is completely covered by an image block, it is marked as repaired; if the character box is only partially covered, it is not marked as repaired.

[0048] S46, see also Figure 2 For each generated image block, a damaged text content image and a damaged text location mask image are created separately.

[0049] S47. Further connected component analysis and morphological processing are performed on the mask image of the damaged text location. Specifically, this includes connecting mask regions with intervals below a preset threshold in both the horizontal and vertical directions, and using a binarization filling algorithm to automatically fill in small holes in the mask. The areas of the newly generated connected regions are further calculated, and only small connected regions with areas below the minimum character frame area threshold are retained and incorporated into the mask area to ensure that the damaged background between character frames can be correctly repaired.

[0050] S48. Save each damaged image (i.e., image block), damaged text content image, damaged text position mask image, and its position information to the repaired image block set. Repeat the above steps until all character boxes are completely repaired and covered.

[0051] S49. For each generated damaged image (i.e., image block), uniformly adjust the size of its damaged text content image and damaged text position mask image to the preset standard size (512×512 pixels) to adapt to the input requirements of the subsequent repair model.

[0052] S410. Convert the resized image block damage image, the mask image of the damaged text location, and the damaged text content image into Tensor format and perform normalization processing. Then, load the above tensor data into a specified computing device (such as a GPU) to improve the computational efficiency of subsequent repair processes.

[0053] S411. Using the repair model trained in step S3, reason about the input damaged image, damaged text content image, and damaged text location mask image to obtain the repaired image block.

[0054] S412. Paste the repaired image block back into the original image according to its actual position in the entire ancient book image. Then, extract the next image block from the set of repaired image blocks generated in S48, and crop it from the updated original image to continue the subsequent repair operations. By repeating the above steps, gradually complete the repair and stitching of all areas to be repaired, and finally obtain the complete repaired ancient book image.

[0055] In summary, compared with the prior art, the method of the present invention has at least the following advantages and beneficial effects: (1) Compared with previous methods that only repaired individual characters in ancient books, the method proposed in this invention can repair damaged ancient book images at the image block level. This method makes full use of intact Chinese characters in the damaged image as style references, thereby generating repaired character images with consistent style and natural integration with the background. At the same time, due to the introduction of a character supervision mechanism, this method shows significant performance advantages, especially in terms of the accuracy of Chinese character repair.

[0056] (2) This invention innovatively proposes an autoregressive method for restoring ancient book images, extending the restoration method limited to image block level to the whole-page level restoration for the first time. By generating damaged text content images and position mask images, and combining them with morphological operations, this method can accurately locate and restore Chinese characters and backgrounds in an entire page of an ancient book, achieving a higher level and a wider range of ancient book image restoration.

[0057] (3) This invention can efficiently restore images of ancient books, and can completely restore an image of an ancient book in just a few minutes. Compared with traditional restoration methods that rely on manual labor and often take several days or even months, this invention greatly improves restoration efficiency, significantly reduces reliance on manual labor, and provides strong technical support for the preservation and digitization of ancient books.

[0058] (4) This invention has good flexibility and portability, and can be easily deployed on operating systems such as Windows 10 and Ubuntu 18.04. In addition, this scheme facilitates the optimization and integration of algorithms, and the overall implementation and application process is simple and efficient.

[0059] Example 2 This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to achieve the following: Figure 4This illustrates a method for restoring length-based images of ancient Chinese books.

[0060] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0061] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, performing various server functions and processing data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by accessing data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or more of a Central Processing Unit (CPU) and a modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0062] Since this electronic device is the electronic device corresponding to the length-level Chinese ancient book image restoration method of this invention, and the principle of solving the problem by this electronic device is similar to that of this method, the implementation of this electronic device can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0063] Example 3 This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to achieve the following: Figure 4This illustrates a method for restoring length-based images of ancient Chinese books.

[0064] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0065] Since this storage medium is the storage medium corresponding to a length-level Chinese ancient book image restoration method in this embodiment of the invention, and the principle of this storage medium in solving the problem is similar to that of this method, the implementation of this storage medium can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0066] Example 4 In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of a length-based Chinese ancient book image restoration method according to various exemplary embodiments of this application as described above. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0067] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0068] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0069] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for restoring length-based images of ancient Chinese books, characterized in that, Includes the following steps: Data preparation: Acquire images of ancient books, and filter out intact and damaged images of ancient books; Data preprocessing: For intact ancient book images, damage is simulated by random masking to construct training data pairs; for damaged ancient book images, the location of missing text is marked and the specific content of the missing text is determined. Incomplete text generation: Construct a restoration model based on a diffusion model, and train the restoration model using the constructed training data to enable the model to learn the writing styles of different ancient books; Autoregressive length-level restoration: Using a trained model, length-level restoration of ancient book images is achieved step by step in the form of overlapping sliding windows through an autoregressive approach. The autoregressive length-level repair includes: Obtain the image of the ancient book to be restored, set the size and sliding step of the image block, initialize the set of character boxes to be restored, and record the area where each character to be restored is located; Enter the autoregressive loop to determine if there are still any unrepaired characters; if all characters have been repaired, terminate the loop; if there are still unrepaired characters, continue with the subsequent steps. Calculate the minimum and maximum x and y coordinates of all unrepaired character boxes to form the minimum bounding rectangle of the unrepaired character boxes; use the four corners of this bounding rectangle as the starting points of candidate image blocks, and calculate the number of unrepaired character boxes completely covered by the image block corresponding to each starting point. Determine the starting point, move the window across the image according to the set sliding step size and direction, and generate image blocks in sequence; whenever a character box is completely covered by an image block, mark it as repaired; For each generated image block, create a damaged text content image and a damaged text location mask image respectively; Connectivity analysis and morphological processing are performed on the mask image of the damaged text locations; Save each image block, the damaged text content image, the damaged text location mask image and its location information to the repaired image block set until all character boxes are completely repaired and covered. For each generated image block, the size of its damaged text content image and damaged text position mask image are uniformly adjusted to the preset standard size; The image block, the image of the damaged text content, and the mask image of the location of the damaged text are input into the restoration model for inference to obtain the restored image block; Paste the restored image blocks back into the original image according to their actual positions within the entire ancient book image.

2. The method for restoring length-based images of ancient Chinese books according to claim 1, characterized in that, The data preprocessing includes: For intact ancient book images, the ancient book images are cut into image blocks of a preset size, and the text areas of the image blocks are randomly destroyed by random masking to form "intact-damaged" data pairs; For damaged images of ancient books, the positions of the missing text are marked using character boxes, and the marking results are recorded in a uniform format; For the marked locations of missing characters, the content of each missing character is restored to obtain an image with the missing characters repaired.

3. The method for restoring length-based images of ancient Chinese books according to claim 1, characterized in that, The restoration model takes the damaged image, the mask image of the damaged location, and the image of the damaged character content as input, and generates a restoration result of the ancient book image that is consistent in character style and background. The repair model includes a text location and content decoder and a U-shaped backbone network; The text location and content decoder is used to generate a corresponding mask image of the damaged text location and a damaged text content image based on the marked damaged text location and content. The input of the U-shaped backbone network consists of the original damaged image, the damaged text content image, the damaged text location mask image, and a noise image, and the output is the repaired image.

4. The method for restoring length-based images of ancient Chinese books according to claim 1, characterized in that, The repair model is trained using a character-supervised mechanism, with the loss function being: In the formula, Indicates VGG network, To repair the image, For the target image, To repair the region mask image.

5. The method for restoring length-based images of ancient Chinese books according to claim 1, characterized in that, The determination of the starting point includes: Select the candidate image patch with the fewest unrepaired character boxes as the starting point of the sliding window.

6. The method for restoring length-based images of ancient Chinese books according to claim 1, characterized in that, The connected component analysis and morphological processing of the mask image of the damaged text location includes: In the horizontal and vertical directions, mask regions with intervals lower than a preset threshold are connected, and a binary filling algorithm is used to automatically fill in small holes in the mask. The area of ​​newly generated connected regions is calculated, and only small connected regions with an area lower than the minimum character box area threshold are retained and incorporated into the mask area to ensure that the broken background between character boxes can be correctly repaired.

7. The method for restoring length-based images of ancient Chinese books according to claim 1, characterized in that, Before inputting the image patch, the damaged text content image, and the damaged text location mask image into the restoration model, the following steps are also included: The resized image blocks, the mask image of the damaged text location, and the damaged text content image are converted into Tensor format and then normalized.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method as described in any one of claims 1 to 7.

Images