A method for digitizing ancient documents
By constructing an ancient book recognition system that integrates sequence recognition, semantic analysis, and inter-class difference learning techniques, the problem of low recognition accuracy in ancient books has been solved, achieving efficient and automated recognition of ancient books, reducing the need for manual annotation, and promoting the digitization process of ancient books.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TONGFANG KNOWLEDGE DIGITAL PUBLISHING TECH CO LTD
- Filing Date
- 2022-12-02
- Publication Date
- 2026-08-04
AI Technical Summary
Existing OCR technology suffers from low accuracy in recognizing ancient books and documents. This is mainly due to issues such as paper fading, deterioration, damage, seals obscuring text, and font adhesion in ancient books and documents, which make it difficult for deep learning training to converge. Furthermore, there is a lack of sufficient data support, and manual annotation is inefficient.
An ancient book recognition system was constructed, integrating sequence recognition, semantic analysis, and inter-class difference learning techniques. It uses an ultra-lightweight model structure, a bidirectional multi-gate recurrent network, and a self-attention model, combined with a multi-stage, multi-task training strategy, to build an ancient book text dataset and perform self-learning to improve recognition accuracy.
It has achieved efficient and automated recognition of ancient books and documents, reduced the need for manual annotation, improved the recognition accuracy, and promoted the digitization process of ancient books.
Smart Images
Figure CN115797946B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of ancient book document digitization technology, and in particular to a method for digitizing ancient book documents. Background Technology
[0002] Traditional ancient books and documents are precious historical and cultural heritages, containing the wisdom and essence of my country's five-thousand-year history, and possess significant scientific and cultural value. Ancient books and documents are non-renewable cultural resources; therefore, digitization is an important way to preserve these documents and pass on culture. Manually processing massive amounts of ancient books and documents is not only time-consuming and laborious but also prone to errors. Therefore, it is necessary to use computers to automate the processing of images of ancient books.
[0003] The textual expressions in ancient books are extremely diverse, including mixed horizontal and vertical text, characters of different sizes, incomplete or blurred text, yellowed and aged paper, and stains on the pages. Therefore, image analysis and recognition of ancient books is a very challenging task.
[0004] Digitizing ancient texts has been an ongoing process, but it has proven difficult and costly. Traditionally, the input of ancient texts relies heavily on manual labor, which is both labor-intensive and extremely inefficient. Even with OCR technology heavily utilizing deep learning, applications of ancient text digitization still face the challenge of a lack of rigorous and sufficient data. Conventional OCR methods are highly manual, requiring professionals to annotate each character individually, hindering the scalability of ancient text digitization and resulting in a slow overall progress.
[0005] Existing OCR technology primarily addresses text recognition in modern scenarios, such as documents, newspapers, and forms. In scanned images of modern books at resolutions above 300 DPI, existing OCR technology can achieve a 99.9% accuracy rate for printed text, provided there are no borders or noise interference, the characters are clear, and there are no obvious merging or missing strokes. However, this technology has drawbacks. Ancient texts are typically quite old, often exhibiting paper fading, deterioration, or damage. Complex layouts also frequently suffer from issues like seals obscuring text, font merging, and missing characters, all of which significantly impact OCR accuracy. Compared to commonly used modern characters, ancient texts contain a much richer variety of characters, including numerous traditional and variant forms, more easily confused similar characters, and diverse writing styles—all significant factors affecting the convergence of deep learning training. Therefore, the accuracy of existing OCR technology drops significantly when applied to ancient text recognition.
[0006] In the traditional process of digitizing ancient books, the text input is usually done manually, which is labor-intensive and extremely inefficient. Even with OCR technology heavily reliant on deep learning, ancient book digitization applications still face the problem of a lack of rigorous and sufficient data. Conventional OCR methods are highly manual, requiring professionals to annotate each character one by one, which prevents the digitization of ancient books from achieving economies of scale and results in a slow overall process. Summary of the Invention
[0007] To address the aforementioned technical problems, the purpose of this invention is to provide a method for digitizing ancient books and documents.
[0008] The objective of this invention is achieved through the following technical solution:
[0009] A method for digitizing ancient books and documents includes:
[0010] Step A: Obtain image data of individual characters from ancient books;
[0011] Step B constructs an ancient text dataset using the acquired image data;
[0012] Step C: Integrate sequence recognition, semantic analysis, and inter-class difference learning techniques to construct an ancient book recognition system;
[0013] Step D involves identifying ancient texts using an ancient text recognition system.
[0014] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:
[0015] This method proposes a new approach to the digitization of ancient books and documents. Based on a calligraphy dictionary, it constructs a dataset of ancient book texts containing various calligraphy styles and builds a brand-new ancient book recognition system to recognize ancient documents, thus promoting the digitization of ancient books and documents and providing strong support for the research of ancient texts. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the methods for digitizing ancient books and documents;
[0017] Figure 2 This is a sampled instance graph of the dataset used;
[0018] Figure 3 This is a flowchart illustrating the method for constructing ancient text datasets;
[0019] Figure 4 This is a schematic diagram of a generated image instance of an ancient book at the length level;
[0020] Figure 5 This is a schematic diagram of the ancient book recognition model structure;
[0021] Figure 6 It is a flowchart of multi-stage and multi-task training of an ancient book recognition model;
[0022] Figure 7 It is an example diagram of the result obtained by the method of digitizing ancient book documents. Specific implementation manners
[0023] To make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with embodiments and accompanying drawings.
[0024] As Figure 1 shown, it is a method flow of digitizing ancient book documents, including
[0025] Step 10: Obtain ancient book single-character image data;
[0026] The fonts of Chinese calligraphy include: oracle bone script, great seal script (zhou script, bronze script / zhongding script, stone drum script, miao zhuan, bird and insect script, etc.), small seal script (yu jin script, yu zhu script), official script (fen shu / ba fen shu, Han bamboo slips, Han official script, etc.), cursive script (zhang cao, xiao cao, da cao, standard cursive script, modern cursive script, wild cursive script), regular script (zheng shu / zhen shu, Wei tablet), running script (xing kai, xing cao), thin gold script, Mao script, flying white script, etc. 5,008,510 pictures are obtained from a calligraphy dictionary, including a total of 14,698 character categories, and each character category contains multiple above-mentioned calligraphy fonts.
[0027] An example diagram of the dataset sampling used is as Figure 2 shown (the example shows the regular script, running script, cursive script, official script, and seal script fonts of the character "book").
[0028] Step 20: Construct an ancient book character dataset through the obtained image data;
[0029] Constructing an ancient book character dataset is to generate ancient book character images with single-column annotations at the volume level, specifically including single-character image preprocessing, single-character image annotation, ancient book text collation, and volume image generation. The specific process is as Figure 3 shown.
[0030] (1) Single-character image processing:
[0031] Use image processing technology to obtain a binary image of the ancient book character image.
[0032] Due to influencing factors such as shooting environment and acquisition method, there are problems such as noise and inconsistent contrast in the character image. First, use feature functions such as color, morphology, and texture to process the picture to remove the background; during the background removal operation, the effect of the text part to be retained may be weakened, so it is necessary to enhance the intensity of the retained text part again.
[0033] Image processing of ancient texts involves further processing of the acquired text images. Image processing techniques, combined with the HSV color space method and brightness equalization processing algorithm, are used to perform primary noise reduction on the images. The color images are converted to black and white images through grayscale conversion, and then binarization calculations are performed to highlight the edges of the text.
[0034] (2) Single-word image annotation:
[0035] Single-character image annotation involves processing and storing the image files of a single character obtained in (1) according to certain naming rules, and integrating text images with the same label and font into the same folder.
[0036] (3) Compilation of ancient texts
[0037] Ancient text collation involves organizing the texts of a number of important ancient books, including Records of the Grand Historian, Book of Han, Book of the Later Han, Records of the Three Kingdoms, Book of Jin, Romance of the Three Kingdoms, Dream of the Red Chamber, The Scholars, Dream Pool Essays, and A New Account of Tales of the World, to create realistic tags for generating full-length images of ancient texts in later stages.
[0038] (4) Image generation
[0039] The format image generation is based on ancient texts and generates format-level ancient text images with single-column annotations. For each ancient text, multiple format-level ancient text images can be generated to correspond to it, presented in various forms such as regular script, running script, cursive script, clerical script, and seal script.
[0040] A schematic diagram of an example of a generated image of an ancient book at the length level is shown below. Figure 4 The example shown is an image of the regular script text from the "Annals of the Five Emperors" section of the Records of the Grand Historian, specifically the "Twelve Annals" section. The text reads: "The Yellow Emperor was the son of Shaodian, surnamed Gongsun, and named Xuanyuan."
[0041] Step 30: Integrate sequence recognition, semantic analysis, and inter-class difference learning techniques to construct an ancient book recognition system;
[0042] Building upon traditional OCR recognition technology, a novel ancient book recognition system has been constructed to address technical bottlenecks. This system integrates machine learning techniques such as sequence recognition, semantic analysis, and inter-class difference learning to create a recognition system that enables iterative self-learning of the model.
[0043] (1) Ultralight model structure
[0044] The ancient book recognition system employs an ultra-lightweight model structure, offering significant advantages in inference speed and parameter count, while achieving a good balance between speed and accuracy. Drawing inspiration from classic residual model unit structures, it introduces channel obfuscation, grouped convolution, and depthwise separable convolution. This ensures sufficient model depth while actively expanding its width, enabling information exchange between channels and learning inter-channel dependencies while maintaining the same channel dimension. The fusion architecture between modular units within the model structure also achieves multi-scale feature fusion operations within the model, reducing the overall parameter count and computational cost while enhancing the model's adaptability to complex backgrounds and diverse text. A schematic diagram of the model structure is shown below. Figure 5 As shown.
[0045] (2) Sequence recognition model combining semantic information
[0046] The ancient book recognition system model, based on sequence recognition structure, incorporates semantic information to enhance text recognition results. The system combines a bidirectional multi-gate recurrent network with a bidirectional self-attention model. By jointly regulating the preceding and following neurons in all neural layers within the network, it performs deep bidirectional representation learning on the graphic and image features extracted by the convolutional network. This achieves dimensionality reduction and semantic encoding of image features. Different levels simultaneously perform multiple rounds of iterative learning on different semantic dimensions, thereby mining the semantic representation and association information contained within the graphic and image features.
[0047] (3) Multi-stage multi-task training strategy
[0048] During the training of the recognition model within the system, a multi-stage, multi-task strategy is employed to ensure the model's accuracy in recognizing similar-looking characters, traditional characters, and variant characters, as well as its ability to overcome complex backgrounds. In the initial training phase, the model is trained using a mixture of synthetic and real data. This phase uses only a single task as the model's objective. The advantage of this is that synthetic samples and features cannot perfectly match real sample features; maintaining a certain proportion of real samples in the training samples helps neutralize the feature distribution bias caused by synthetic samples, facilitating rapid model convergence. After completing the initial training phase, the trained model is used to recognize and calculate the core representation for each character category, which is then applied to the second phase. In the second phase, the model is fine-tuned using real data samples. The training task not only ensures the correctness of sequence recognition but also ensures that the learned character representations are closest to the core representations of their respective categories and further away from the core representations of other categories. After the second phase of training, the system's recognition results are fed back to professionals, and the corresponding feedback is injected into the system to help it identify text categories that have not been adequately learned. The system will then expand and selectively filter artificially synthesized data and real samples to begin the training process again, forming an effective closed-loop self-learning process. The process is as follows: Figure 6 As shown, the training samples extracted from the original data will be used to train the ancient book recognition model in two stages, and the training effect will be judged. If the training effect meets the standard, it will be fed back to professionals. If the training effect does not meet the standard, the samples will be classified into difficulty levels. For simple samples, data augmentation will be performed in a combination of methods, including spatial geometric transformation, Gaussian blur, color jitter, contrast enhancement and partial occlusion. Then, the augmented samples will be trained. For difficult samples, augmented samples will be automatically generated based on the difficult samples and then trained.
[0049] Examples of results obtained from methods for digitizing ancient books and documents are as follows: Figure 7 As shown.
[0050] Step 40: Identify ancient documents using an ancient book identification system.
[0051] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.
Claims
1. A method for digitizing ancient documents, characterized by, The method includes the following steps: Step A: Obtain image data of individual characters from ancient books; Step B constructs an ancient text dataset using the acquired image data; Step C: Integrate sequence recognition, semantic analysis, and inter-class difference learning techniques to construct an ancient book recognition system; Step D involves identifying ancient texts using an ancient text recognition system; The ancient text dataset includes 1,995,107 images obtained from calligraphy dictionaries, covering oracle bone script, large seal script, small seal script, clerical script, cursive script, regular script, running script, thin gold script, Mao script, and flying white script. It includes 4,637 text categories, with each text category containing multiple calligraphy fonts. Step B specifically includes: Step B1 preprocesses the single-character images to obtain binary images of the ancient text images; Step B2: Single-word image annotation; Step B3 involves organizing the ancient texts; Step B4 generates a text image of the ancient book with single-column annotations based on the ancient book text; Step B31 adopts an ultra-lightweight model structure, which introduces channel confusion, grouped convolution and depthwise separable convolution into the residual model unit structure to form an ultra-lightweight model structure. Step B32: The ancient book recognition system introduces a sequence ancient book recognition model with semantic information based on the sequence recognition structure. Step B33 uses a multi-stage, multi-task training strategy to train the ancient book recognition model within the ancient book recognition system. Step B33 includes an initial training phase and a second training phase. The initial training phase involves training the ancient book recognition model by mixing synthetic and real data. The second training phase involves identifying and calculating the core representation of each text category using the ancient book recognition model trained in the initial phase, fine-tuning the ancient book recognition model using real data samples, so that the learned text representations are closest to the core representations of the same text category and farther away from the core representations of different text categories, and then feeding the recognition results back to professionals. The ancient book recognition system adopts an ultra-lightweight model structure, which draws on the classic residual model unit structure and introduces channel confusion, grouped convolution and depthwise separable convolution to realize information interaction between channels and learn the dependencies between channels. The ancient book recognition system model, based on the sequence recognition structure, introduces semantic information to help improve the text recognition results. The system combines a bidirectional multi-gate recurrent network and a bidirectional self-attention model. By jointly regulating the preceding and following neurons in all neural layers of the network, it performs deep bidirectional representation learning on the graphic and image features extracted by the convolutional network, realizing the function of dimensionality reduction semantic encoding of image features. Its different levels simultaneously perform multiple rounds of iterative learning on different semantic dimensions, thereby mining the semantic representation information and association information contained in the graphic and image features. During the training of the recognition model within the system, a multi-stage, multi-task strategy is employed. In the initial training stage, the model is trained using a mixture of synthetic and real data. After completing the initial training stage, the trained model is used to recognize and calculate the core representation of each character category, which is then applied to the second stage. In the second stage, real data samples are used to fine-tune the model. After the second stage training is completed, the system's recognition results are fed back to personnel, and the corresponding feedback is injected into the system to help it identify character categories that have not been adequately learned. The system will then expand and selectively filter artificially synthesized data and real samples, and restart the training process, forming an effective closed-loop self-learning process. In step B32, the ancient book recognition system combines a bidirectional multi-gate recurrent network with a bidirectional self-attention model. By jointly regulating the neurons before and after the convolutional network, it performs deep bidirectional representation learning on the graphic and image features extracted by the convolutional network, thereby achieving dimensionality reduction semantic encoding of image features. Different levels simultaneously perform multiple rounds of iterative learning on different semantic dimensions, thereby mining the semantic representation information and association information contained in the graphic and image features. The multi-stage, multi-task training of the ancient book recognition model specifically includes: training the ancient book recognition model in two stages using training samples extracted from the original data and judging the training effect. If the training effect meets the standard, feedback is given to professionals; if the training effect does not meet the standard, the samples are classified into difficulty levels. For simple samples, data augmentation is performed using a combination of methods, including spatial geometric transformation, Gaussian blur, color dithering, contrast enhancement, and partial occlusion, and then the augmented samples are trained. For difficult samples, augmented samples are automatically generated based on the difficult samples and then trained.
2. The method of digitizing a historical document as claimed in claim 1, wherein, Step B1 includes: Image processing and background removal are performed using color, shape, and texture feature functions; Image processing techniques, combined with the HSV color space method and brightness equalization processing algorithm, are used to perform primary noise reduction on the image. The color image is converted into a black and white image by grayscale conversion and then binarized.
3. The method of digitizing a historical document as claimed in claim 1, wherein, Step B2 involves processing and storing the acquired single-character image files according to certain naming rules, and grouping text images with the same label and font into the same folder.
4. The method of digitizing a historical document as claimed in claim 1, wherein, The compiled ancient texts are used as real labels to generate lengthy images of ancient texts.