Handwritten Manchu identification method for historical Manchu archives

Abstract

The invention discloses a handwritten Manchu identification method for historical Manchu archives, and belongs to the field of images. A conventional Manchu identification method in a Manchu file image has an error segmentation situation, which affects the accuracy of a subsequent identification result. The method comprises the following steps: segmenting a Manchu history file image by using a trained Unet network, obtaining a column ridge line image, projecting the column ridge line image on a horizontal axis to form a projection curve, obtaining the Manchu column number in the Manchu file image through wave crest and wave trough detection, and obtaining a center coordinate of the projection image; calculating the average column width of the column ridge line image; designing a handwritten Manchu text column recognition model and a data set, wherein the recognition model comprises a column feature encoder and a column feature decoder; and intercepting a text column area image from an original to-be-recognized image, sending the text column area image into the trained handwritten Manchu text column recognition model, and outputting a recognition result in a Latin mode. According to the method, the handwritten Manchu column text in the Manchu historical archive image is recognized without segmentation by adopting the deep learning technology, and the recognition accuracy is improved.

Description

Technical Field

[0001] The invention relates to a handwritten Manchu recognition method, and in particular to a handwritten Manchu recognition method oriented to Manchu historical archives. Background Art

[0002] Manchu historical archives are important materials for studying the history of the Ming and Qing dynasties and Manchu culture. Due to repeated use, Manchu historical archives have been damaged and polluted to varying degrees. In addition to preserving historical materials, since fewer and fewer people can speak Manchu, it is urgent to protect Manchu culture. Digitizing Manchu historical archives can effectively use modern means such as computer technology, database technology, and artificial intelligence technology to protect Manchu historical documents and provide a variety of research methods. Optical character recognition (OCR) technology is one of the core technologies for digitizing historical archives.

[0003] The vast majority of Manchu historical archives are handwritten, and there are problems such as free writing form, adhesion between columns, close distance between words, and breaks between words, which affect the segmentation of Manchu historical archive images. The Manchu recognition method based on segmentation is to divide the document into columns first and then into words, which will produce cumulative errors and seriously affect the Manchu recognition accuracy of Manchu historical archive images.

[0004] The present invention is aimed at the recognition of handwritten Manchu in Manchu historical archive images, and adopts deep learning technology to realize the recognition of handwritten Manchu columns in non-segmented Manchu historical archive images. It is one of the key supporting technologies for the digitization of Manchu historical archives. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that in the existing recognition process of handwritten Manchu in Manchu historical archive images, the image needs to be divided into columns and words for Manchu recognition, which produces cumulative errors and affects the Manchu recognition accuracy of the Manchu historical archive images, and to propose a handwritten Manchu recognition method for Manchu historical archives.

[0006] The above purpose is achieved through the following technical solutions: A method for recognizing handwritten Manchu characters for Manchu historical archives, the method comprising: A method for recognizing handwritten Manchu characters for Manchu historical archives, the method comprising: Step 1: Create a network training data set to train the Unet network, use the trained Unet network to segment the Manchu historical archive images, and obtain the column ridge image. Unet uses VGG16 as the backbone network; The process of making the network training data set is: opening and reading the Manchu historical archive image, manually marking the column ridges on the Manchu historical archive image using a red line drawing tool and saving it as a column ridge marking map, opening the column ridge marking map through the OpenCV library and separating the channels, binarizing the red channel to form a binary column ridge marking map; then, randomly shearing the binary column ridge marking map and the original image to be identified according to the size of the area of ​​423×576, and using random rotation to perform data enhancement to make a data set; Step 2: Project the column ridge image obtained in step 1 onto the horizontal axis to form a projection curve, and obtain the number of Manchu columns contained in the Manchu document image through peak and valley detection. L , and obtain the center coordinates of the projected image x i , all center coordinates are expressed as a set as { x 1 , x 2 ,… x i …, x L}; Step 3: Calculate the average column width of the column ridge image using the following formula: w , specifically: , Step 4: Extract the text column area image from the original image to be recognized, specifically: In the original image to be recognized, the point ( x i - w / 2, 0) is the upper left vertex, point ( x i + w / 2, h) is the rectangular area of ​​the lower right vertex, where h represents the image height; Step 5: Design a handwritten Manchu text column recognition model, which includes a column feature encoder CNN_simAm and a column feature decoder BiLSTM; and design a data set for model training to train the handwritten Manchu text column recognition model; wherein the column feature encoder CNN_simAm encoder is expressed as: Conv+Mish+Conv+Mish+InstanceNormal2d+Conv+simAM; Mish is the activation function, and its expression is: f ( x )= x * tanh ( ln (1+ e x)); simAM stands for Similarity-based Attention Module, which means a parameter-free attention module based on similarity; the structure of the handwritten Manchu text column recognition model is as follows: , Step 6: The text column area image obtained in step 4 is sent to the handwritten Manchu text column recognition model trained in step 5, and the recognition result is output in Latin transcription.

[0007] Furthermore, the column feature encoder CNN_simAm described in step five is composed of an adaptive maximum pooling layer AdaptiveMaxPool2d and a convolution layer Conv2d, and the convolution layer Conv2d has an input channel of 512 and an output channel of 121.

[0008] Furthermore, the step of designing a data set for model training described in step five is to use the labelme annotation tool to frame the text column area image, and organize the annotation text in the form of rows in the text file to facilitate the reading of the annotation text information and model training, so that the annotation text information is structured, and each row corresponds to 1 column of images in the image, which is used for the data loading program to cut the image in the annotated rectangular frame and read the corresponding annotation text during the training process.

[0009] Furthermore, the process of training the handwritten Manchu text column recognition model described in step five adopts a hybrid Dropout strategy, that is, randomly adopting the Dropout function and the dropout2d function for training processing, and the processing flow is: generate a random number [0,1], and determine whether the random number is greater than 0.5, if it is greater than 0.5, execute the Dropout function, otherwise execute the dropout2d function.

[0010] The beneficial effects of the present invention are: 1. The principle of the previous segmentation-based Manchu recognition method is to first segment the document into columns and then into words. However, there are errors in the segmentation process, such as Figure 1 As shown, this will seriously affect the accuracy of subsequent recognition results. The present invention is oriented to Manchu historical archive images, and adopts deep learning technology to realize the recognition of handwritten Manchu columns in non-segmented Manchu historical archive images, and produces a data set for training the Unet network to improve the training effect. Overcome the cumulative error caused by handwritten Manchu recognition based on word segmentation method, adopt deep learning technology to realize the recognition of handwritten Manchu columns in non-segmented Manchu historical archive images, and improve recognition accuracy. It is one of the key supporting technologies for the digitization of Manchu historical archives.

[0011] 2. The method of the present invention designs a unique data set for training a handwritten Manchu text column recognition model, and adopts a hybrid Dropout strategy to train the handwritten Manchu text column recognition model to improve the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a diagram showing a situation in which errors exist in the existing handwritten Manchu word segmentation involved in the present invention; Figure 2 The present invention relates to a Manchu historical archive image to be identified; Figure 3 is a flow chart of the method of the present invention; Figure 4 It is an example of a data set for training a Unet network involved in the present invention; Figure 5 This is an example of a data set for training a handwritten Manchu text column recognition model involved in the present invention; Figure 6 The Unet network of the present invention is used to extract the ridge line effect diagram of the Manchu column; Figure 7 The present invention relates to a Manchu text column image cropped based on the extracted column ridge line; Figure 8 The present invention relates to a handwritten Manchu text column recognition model structure. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Specific implementation method one:

[0015] The present embodiment provides a method for recognizing handwritten Manchu characters for Manchu historical archives, such as Figure 3 As shown, the method includes: Step 1: Make Figure 4 The network training data set shown is used to train the Unet network, and the trained Unet network is used to segment the Manchu historical archive images to obtain Figure 6 The column ridge image shown, where Unet uses VGG16 as the backbone network; The process of making the network training data set is: opening and reading the Manchu historical archive image, using a red line drawing tool to manually mark the column ridges on the Manchu historical archive image and save it as a column ridge marking map, opening the column ridge marking map through the OpenCV library and separating the channels, binarizing the red channel to form a binary column ridge marking map; then, randomly shearing the binary column ridge marking map and the original image to be identified according to the size of the area of ​​423×576, and using random rotation to perform data enhancement to make a data set to improve the training effect; Step 2: Project the column ridge image obtained in step 1 onto the horizontal axis to form a projection curve, and obtain the number of Manchu columns contained in the Manchu document image through peak and valley detection. L , and obtain the center coordinates of the projected image x i , all center coordinates are expressed as a set as { x 1 , x 2 ,… x i …, x L}; Step 3: Calculate the average column width of the column ridge image using the following formula: w , specifically: , Step 4: Extract the following from the original image to be recognized: Figure 7 The text column area image shown is: In the original image to be recognized, the point ( x i - w / 2, 0) is the upper left vertex, point ( x i + w / 2, h) is the rectangular area of ​​the lower right vertex, where h represents the image height; Step 5: Design a handwritten Manchu text column recognition model. The model structure is as follows: Figure 8 As shown, the handwritten Manchu text column recognition model includes a column feature encoder CNN_simAm and a column feature decoder BiLSTM; and is designed as follows Figure 5 The data set used for model training is shown in Figure 1 to train a handwritten Manchu text column recognition model; the column feature encoder CNN_simAm encoder is represented as: Conv+Mish+Conv+Mish+InstanceNormal2d+Conv+simAM; Mish is the activation function, and its expression is: f ( x )= x * tanh ( ln (1+ e x )); simAM stands for Similarity-based Attention Module, which means a parameter-free attention module based on similarity; the structure of the handwritten Manchu text column recognition model is as follows: , Step 6: The text column area image obtained in step 4 is sent to the handwritten Manchu text column recognition model trained in step 5, and the recognition result is output in Latin transcription. Specific implementation method 2:

[0017] Different from the first specific implementation mode, in this implementation mode, a handwritten Manchu recognition method for Manchu historical archives is used. The column feature encoder CNN_simAm described in step five is composed of an adaptive maximum pooling layer AdaptiveMaxPool2d and a convolutional layer Conv2d. The convolutional layer Conv2d has an input channel of 512 and an output channel of 121. Specific implementation method three:

[0019] Different from the first or second specific implementation modes, in this implementation mode, a handwritten Manchu recognition method for Manchu historical archives is used. The step of designing a data set for model training described in step five is to use the labelme annotation tool to frame the text column area image, and organize the annotated text in the form of rows in the text file to facilitate the reading of the annotated text information and model training, so that the annotated text information is structured, and each row corresponds to one column of images in the image, which is used for the data loading program to cut the annotated rectangular frame image and read the corresponding annotated text during the training process. Specific implementation method four:

[0021] Different from the third specific implementation method, in this implementation method for handwritten Manchu recognition of Manchu historical archives, in order to improve the generalization ability of the model, the process of training the handwritten Manchu text column recognition model described in step five adopts a hybrid Dropout strategy, that is, randomly adopts the Dropout function and the dropout2d function for training processing, and the processing flow is: generate a random number [0,1], and determine whether the random number is greater than 0.5, if it is greater than 0.5, execute the Dropout function, otherwise execute the dropout2d function.

[0022] The embodiments of the present invention disclose preferred embodiments, but are not limited thereto. A person skilled in the art can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes. However, as long as they do not deviate from the spirit of the present invention, they are all within the protection scope of the present invention.

Claims

1. A method for recognizing handwritten Manchu characters in Manchu historical archives, the method comprising: Step 1: Create a network training data set to train the Unet network, and use the trained Unet network to segment the Manchu historical archive images to obtain the column ridge images, where: Unet uses VGG16 as the backbone network; The process of making the network training data set is: opening and reading the Manchu historical archive image, manually marking the column ridges on the Manchu historical archive image using a red line drawing tool and saving it as a column ridge marking map, opening the column ridge marking map through the OpenCV library and separating the channels, binarizing the red channel to form a binary column ridge marking map; then, randomly shearing the binary column ridge marking map and the original image to be identified according to the size of the area of ​​423×576, and using random rotation to perform data enhancement to make a data set; Step 2: Project the column ridge image obtained in step 1 onto the horizontal axis to form a projection curve, and obtain the number of Manchu columns contained in the Manchu document image through peak and valley detection. L , and obtain the center coordinates of the projected image x i , all center coordinates are expressed as a set as { x 1 , x 2 ,… x i …, x L }; Step 3: Calculate the average column width of the column ridge image using the following formula: w , specifically: ， Step 4: Extract the text column area image from the original image to be recognized, specifically: In the original image to be recognized, the point ( x i - w / 2, 0) is the upper left vertex, point ( x i + w / 2, h) is the rectangular area of ​​the lower right vertex, where h represents the image height; Step 5: Design a handwritten Manchu text column recognition model, which includes a column feature encoder CNN_simAm and a column feature decoder BiLSTM; and design a data set for model training to train the handwritten Manchu text column recognition model; wherein the column feature encoder CNN_simAm encoder is expressed as: Conv+Mish+Conv+Mish+InstanceNormal2d+Conv+simAM; Mish is the activation function, and its expression is: f ( x )= x * tanh ( ln (1+ e x )); simAM stands for Similarity-based Attention Module, which means a parameter-free attention module based on similarity; the structure of the handwritten Manchu text column recognition model is as follows: ， Step 6: The text column area image obtained in step 4 is sent to the handwritten Manchu text column recognition model trained in step 5, and the recognition result is output in Latin transcription.

2. According to the method for handwritten Manchu recognition for Manchu historical archives described in claim 1, the column feature encoder CNN_simAm described in step 5 is composed of an adaptive maximum pooling layer AdaptiveMaxPool2d and a convolutional layer Conv2d, and the convolutional layer Conv2d has an input channel of 512 and an output channel of 121.

3. According to the handwritten Manchu recognition method for Manchu historical archives described in claim 2, the step of designing a data set for model training described in step 5 is to use the labelme annotation tool to frame the text column area image, and organize the annotated text in the form of rows in the text file to facilitate the reading of the annotated text information and model training, so that the annotated text information is structured, and each row corresponds to 1 column of images in the image, which is used for the data loading program to cut the annotated rectangular frame image and read the corresponding annotated text during the training process.

4. According to the method for handwritten Manchu recognition for Manchu historical archives described in claim 3, the process of training the handwritten Manchu text column recognition model described in step 5 adopts a hybrid Dropout strategy, that is, randomly adopting the Dropout function and the dropout2d function for training processing, and the processing flow is: generate a random number [0,1], and determine whether the random number is greater than 0.5, if it is greater than 0.5, execute the Dropout function, otherwise execute the dropout2d function.

Images