A method for woodblock printing Manchu recognition based on end-to-end neural network

Abstract

The present invention provides a method for recognizing Manchu characters in block prints based on an end-to-end neural network. The method comprises: extracting independent Manchu words from images of ancient Manchu books, representing each Manchu word with a Manchu word image; extracting feature vectors of the Manchu word images by performing a convolutional neural network (CNN) on the Manchu word images; processing the feature vectors extracted by the CNN using a long short-term memory (LSTM) network to obtain a vector sequence; and converting the vector sequence output by the LSTM into a label sequence corresponding to the Manchu words using a connectivity temporal classification (CTC) algorithm, and recognizing the Manchu words based on the label sequence. The method can effectively extract and identify clear, separable, and universal features from a large number of printed Manchu images, while also effectively screening and correlating relevant feature information in the context, thereby obtaining a printed Manchu recognition model with high recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of text recognition, and in particular to a method for recognizing Manchu characters printed on woodblock prints based on an end-to-end neural network. Background Art

[0002] Current methods for text recognition in the field of natural language processing (NLP) primarily encompass four steps: text localization, text verification, text input, and text recognition. The goal of text localization is to accurately locate the various text elements contained in an image. The localization process reduces the presence of non-text characters in the image to facilitate subsequent text verification, extraction, and recognition in the located area. Text localization methods primarily include gradient transformation analysis, stroke transformation analysis, connected component analysis, and neural network-based methods. Text verification aims to verify whether the located image area contains the text to be recognized, thereby reducing the probability of the subsequent network model identifying non-text. Text verification methods primarily include support vector machines (SVMs) and convolutional neural networks (CNNs). Text input involves inputting verified text information from an image into the network model to complete text recognition. Currently, text input has become one of the most challenging problems in text recognition. Text input methods are primarily categorized as segmentation-based and non-segmentation-based. Segmentation-based methods are further divided into line-level segmentation and character-level segmentation. In segmentation-based methods, the accuracy of text recognition depends largely on the effectiveness of the specific segmentation. Text recognition primarily converts the image text information input into the network into a target character sequence. This is the final step in the entire method and determines the recognition performance of the entire network model. Currently, text recognition methods primarily include traditional (non-deep learning-based) and deep learning-based methods.

[0003] Currently, existing text recognition methods include manual recognition and machine learning. Manual recognition typically requires manual extraction of text features from images. This method relies on manually extracting clear features, resulting in low generalization and performance. Hidden Markov Models (HMMs) are a common method in traditional machine learning, often used for text recognition and error analysis.

[0004] With the development of deep learning, deep neural networks (DNNs) have been introduced as an effective new method in text recognition, overcoming some of the shortcomings of traditional methods. Due to the simple strokes and strong contextual correlation of Mongolian characters, Kang et al. used a long short-term memory (LSTM) network to recognize Mongolian words. Recurrent neural networks (RNNs) rely on previous text information and cannot address the long-term dependencies of text. Therefore, LSTMs are used to integrate contextual information. Subsequently, some proposals jointly trained CNNs and RNNs, analyzing the impact of different network types on text recognition accuracy. Other proposals further improved recognition accuracy by using the Connectionist Temporal Classification (CTC) algorithm to convert the RNN output into a target character sequence. Another proposal used an end-to-end neural network model for Mongolian word recognition. This end-to-end model significantly reduces the difficulty of model training and can handle character sequences of arbitrary length.

[0005] Current methods for recognizing Manchu characters from block printing suffer from shortcomings such as manual extraction of word features from Manchu words and the use of statistical methods to classify and identify these features. These methods often rely on manual extraction of clear features from words, leading to issues such as unclear extracted features, high extraction time and cost, and poor generalization.

[0006] Machine learning-based methods can effectively avoid the fuzzy manual feature extraction and achieve better performance, but they can also lead to issues such as ignoring contextual feature relationships and inaccurate feature extraction. Furthermore, recognizing massive Manchu words can still be time-consuming. Traditional machine learning methods such as support vector machines (SVMs) and hidden Markov models (HMMs) simplify classification and regression problems and achieve better performance with small sample sizes. However, they still struggle with identifying multiple similar features when processing massive amounts of data. Summary of the Invention

[0007] The embodiments of the present invention provide a method for recognizing Manchu characters printed on woodblock prints based on an end-to-end neural network, so as to effectively recognize Manchu characters in images.

[0008] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions.

[0009] A method for recognizing Manchu characters printed on woodblock prints based on an end-to-end neural network, comprising:

[0010] Cut out independent Manchu words from the pictures of ancient Manchu books, and represent each Manchu word with a Manchu word picture;

[0011] Use convolutional neural network (CNN) to perform convolution operation on Manchu word images and extract the feature vectors of Manchu word images;

[0012] Use the long short-term memory network (LSTM) to process the feature vectors extracted by CNN to obtain a vector sequence containing semantic information;

[0013] The Connectivity Temporal Classification (CTC) algorithm is used to convert the vector sequence output by LSTM into a label sequence corresponding to Manchu words, and the recognition result of the Manchu words is output according to the label sequence.

[0014] Preferably, the method of cutting out independent Manchu words from the picture of the ancient Manchu book and representing each Manchu word with a Manchu word picture includes:

[0015] Individual Manchu words were cut out from each digitized page of the ancient Manchu book image, and each Manchu word was represented by a Manchu word image with different lengths and widths. All Manchu word images were scaled to the same size and normalized to the same width while maintaining the aspect ratio of the word images. Each Manchu word image was rotated 90 degrees counterclockwise, and different labels were assigned to all Manchu letters. All extracted woodblock-printed Manchu words were annotated with glyph encoding.

[0016] Preferably, when using CNN to perform a convolution operation on the Manchu word image, extracting the feature vector of the Manchu word image includes:

[0017] CNN uses multiple sets of convolution kernels adapted to the feature size of Manchu words, passing the Manchu word images through multiple sets of convolution kernels of different sizes in CNN. The length and width of the convolution kernel represent the size of the feature region extracted by each convolution operation. After each pass through the convolution kernel, the output feature map is reduced in dimensionality using maximum pooling. Maximum pooling selects the maximum value in the candidate region as the overall representation, ultimately obtaining a feature vector containing more word texture information.

[0018] Preferably, the process of using LSTM to process the feature vectors extracted by CNN to obtain a vector sequence includes:

[0019] Use two-directional LSTM to process the feature vector of the Manchu word image and obtain the context information h1,h2,…,h contained in the feature vector T, all context information is integrated to obtain the vector sequence h containing context information output by Bi-LSTM i :

[0020]

[0021] in and Represents the backward and forward hidden state sequences respectively, and the final output vector sequence h i It is the sum of the two.

[0022] Preferably, the Connectivity Temporal Classification (CTC) algorithm is used to convert the vector sequence output by the LSTM into a label sequence corresponding to a Manchu word, and the Manchu word is recognized according to the label sequence, including:

[0023] The CTC algorithm is used to convert the vector sequence output by the LSTM into a label sequence corresponding to the Manchu word. The conditional probabilities between different label sequences are calculated and compared, and the label sequence with the highest conditional probability value is output as the prediction result. The conditional probability of each letter in the Manchu word is defined as the probability of observing the corresponding letter image at a certain moment. The conditional probability of the entire Manchu word is generally obtained by summing the conditional probabilities after permuting different letters.

[0024] The process of calculating the conditional probability of the label sequence by the CTC algorithm includes: 1, y2,....,y r Represented as T, where T is the length of the input sequence, L represents the 36 different character labels used, L' is a blank label added to L, that is, L'=L∪{blank}, where blank represents a blank label, π is defined as any possible label sequence, and there is The conditional probability of π is calculated as follows:

[0025]

[0026] in Indicates that label π appears at time t t The probability of p(π|y) in sequence y 1, y2,....,y r The conditional probability of the label sequence π appearing in ;

[0027] exist A many-to-one mapping method B is defined above, which is described as follows: B first removes duplicate labels in π and removes all blank labels to obtain a new label sequence l. The conditional probability of l is defined as the sum of the conditional probabilities of the label sequence π mapped to l by B.

[0028] The conditional probability calculation formula of l is as follows:

[0029]

[0030] In the no-dictionary mode, the sequence with the highest conditional probability in the label sequence is selected as the Manchu word recognition output. The calculation method is as follows:

[0031]

[0032] h(x)≈B(l′)#(5)

[0033] Where h(x) is the final predicted word label sequence, and l′ represents the label sequence with the highest conditional probability;

[0034] In the dictionary-based mode, the sequence that is closest to the word in the dictionary and has the highest conditional probability is selected as the recognition result output of the Manchu word.

[0035] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the method of the present invention can effectively extract and identify clear, separable and universal features from a large number of printed Manchu images, and at the same time effectively screen and associate the related feature information in the context, thereby obtaining a printed Manchu recognition model with high recognition accuracy.

[0036] Additional aspects and advantages of the present invention will be set forth in part in the following description, will be obvious from the following description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A processing flow chart of a method for recognizing Manchu characters printed on woodblock prints based on an end-to-end neural network according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of different examples of the same Manchu word provided by an embodiment of the present invention.

[0040] Figure 3 A flowchart of using CNN to process Manchu word images is provided in an embodiment of the present invention;

[0041] Figure 4A flowchart of the overall processing of CNN, Bi-LSTM and CTC provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0043] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the description of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or couplings. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0044] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention pertains. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as such herein, will not be interpreted in an idealized or overly formal sense.

[0045] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0046] The embodiment of the present invention introduces deep neural networks into the field of woodblock printing Manchu recognition, avoiding the shortcomings of traditional recognition methods. The embodiment of the present invention combines convolutional neural networks, long short-term memory networks and connectivity temporal classification algorithms to propose an end-to-end neural network-based woodblock printing Manchu recognition method, achieving high-precision recognition of woodblock printing Manchu images. The processing flow of this method is as follows: Figure 1 As shown in the figure, the whole system can be divided into two parts: data preparation module and model training module.

[0047] In the data preparation module, it is necessary to first cut out independent Manchu words from each digitized page of the Manchu ancient book image. Each Manchu word is a PNG format image with different lengths and widths. In order for the network to generate the target string sequence for each word, all Manchu word images need to be scaled to the same size. In the present invention, all word images are normalized to the same width while maintaining the aspect ratio of the word images. Since the writing direction of Manchu is different from that of Chinese, and is written from left to right and from top to bottom, each Manchu word image also needs to be rotated 90 degrees counterclockwise so that it can be correctly recognized and input by the neural network.

[0048] Manchu is a phonetic script composed of Manchu letters (ya). Each Manchu word is composed of different ya, and the ya has different shapes in different positions of the word. Ya can be divided into four forms: independent form, initial form, middle form and final form. In order to enable the model to accurately recognize Manchu words, we assigned different labels to all Manchu letters and annotated all extracted woodblock printed Manchu words with glyph encoding. The specific method of assigning labels refers to the Mulindev transliteration scheme. The Mulindev transliteration scheme was proposed by German linguist Paul Georg von It is the most widely used transliteration scheme in the world. It contains 36 different glyph codes, namely "a", "e", "i", "o", "u", "ū", "n", "ng", "k", "g", "h", "b", "p", "s", "t", "d", "l", "m", "c", "j", "y", "r", "f", "w", "k'", "g'", "h'", "ts'", "ts", "dz", "dzi", "sy", "c'y", "jy".

[0049] Recognizing block-printed Manchu words presents the following challenges:

[0050] (1) The Manchu words in ancient books were printed by engraving on wooden boards. The engraving techniques were diverse and the ink relationships were complex, so the shapes of the same letters and words would be different. Figure 2 Examples of different woodblock printing forms of the same Manchu word are shown.

[0051] (2) Ancient books are stored for a long time, which may cause problems such as paper damage and deformation, and blurred fonts.

[0052] (3) The layout of ancient books is complex, which will affect the cutting and recognition of Manchu words.

[0053] In the model training module, in order to obtain rich feature information in the Manchu word images, we first follow Figure 3The illustrated process uses a CNN to process Manchu word images. The specific steps include: Before entering the CNN convolutional layer, all images are first normalized to the same size. During the CNN convolution operation, the images are passed through multiple sets of convolution kernels of varying sizes. The length and width of the convolution kernels represent the size of the feature region extracted by each convolution operation. Each word image has features of varying sizes. To extract comprehensive feature information from the word image, multiple convolution kernels adapted to the size of the Manchu word features are used. After each convolution kernel pass, max pooling is used to reduce the dimensionality of the output feature map. Max pooling typically selects the maximum value in the candidate region as the overall representation. This operation selects features with better classification recognition while preserving more text texture information. Average pooling, on the other hand, selects the average value in each candidate region as the overall representation, which generally preserves more background information in the image. When extracting features from Manchu word images, it is important to focus on the font itself and filter out irrelevant background information. Therefore, max pooling is combined with CNN convolutional layers.

[0054] The overall processing flow of CNN, Bi-LSTM and CTC provided by the embodiment of the present invention is as follows: Figure 4 As shown. In order to make the extracted feature vector contain contextual information, thereby further improving the accuracy of Manchu word recognition, the present invention uses a long short-term memory network (Long Short-Term Memory, LSTM) to process the feature vector extracted by CNN. The reason for this choice is that the LSTM network is good at dealing with problems and tasks that are sensitive to time series. When faced with serialized data, the LSTM network has a memory effect, and the special gated unit structure it contains can select and utilize effective memory information, thereby improving the long-term dependency problem in RNN. Since the contextual information of letters in Manchu words affects the recognition of the entire Manchu word, a long short-term memory network containing two directions (Bi-directional Long Short-Term Memory, Bi-LSTM) is used to obtain a vector sequence containing contextual information, using h1, h2,…, h T The final output of Bi-LSTM can be expressed as follows:

[0055]

[0056] in and Represents the backward and forward hidden state sequences respectively, and the final output vector sequence h i It is the sum of the two.

[0057] The feature vectors output by the Bi-LSTM are then processed by the Connectionist Temporal Classification (CTC) algorithm and output as the target character sequence corresponding to the Manchu word. The CTC algorithm converts the vector sequence output by the long short-term memory network into a label sequence corresponding to the Manchu word. CTC calculates and compares the conditional probabilities between different sequences and outputs the label sequence with the highest probability as the predicted result, demonstrating remarkable transcription performance and stability. CTC adds blank labels to the existing labels, ignoring the absolute position of each label. The conditional probability of each letter is defined as the probability of observing an image of that letter at a given moment. The conditional probability of the entire word can be obtained by summing the conditional probabilities after permuting and combining different letters.

[0058] The following describes the calculation method of CTC conditional probability. First, the sequence of input CTC is expressed as y 1, y2,....,y r , where T is the length of the input sequence. L represents the 36 different character labels currently used, and L' is a blank label added to L, that is, L'=L∪{blank}, where blank represents a blank label. π is defined as any possible label sequence, and there is The conditional probability of π is calculated as follows:

[0059]

[0060] in Indicates that label π appears at time t t The probability of p(π|y) is expressed in the sequence y1,y2,....,y r The conditional probability of the label sequence π appearing in .

[0061] Next, in A many-to-one mapping method B is defined above, specifically as follows: B first removes duplicate labels from π and then removes all blank labels, resulting in a new label sequence l. For example, B maps "-ara-hh-aa-" to "araha" (where "-" represents a blank label). The conditional probability of l is then defined as the sum of the conditional probabilities of the label sequence π mapped to l by B. Because exponential operations typically consume excessive computer resources, a forward-backward algorithm is often used to efficiently compute Equation 2. The conditional probability of l is calculated as follows:

[0062]

[0063] Finally, a label sequence is selected as the recognition result of the word. There are usually two modes when selecting labels, namely the dictionary-based mode and the dictionary-free mode. In the dictionary-free mode, the sequence with the highest conditional probability in the label sequence is selected as the prediction result output. In the dictionary-based mode, the sequence that is closest to the word in the dictionary and has the highest conditional probability is selected as the prediction result output. However, the accuracy and efficiency of word recognition in this mode are usually limited by the size of the dictionary itself. Therefore, in the present invention, the dictionary-free mode is selected to output the label sequence of the word. The calculation method is as follows.

[0064]

[0065] h(x)≈B(l′)(5)

[0066] Where h(x) is the final predicted word label sequence, and l′ represents the label sequence with the highest conditional probability.

[0067] The experiments used in this paper were completed in the Ubuntu 18.04 operating system, Tensorflow version 1.13.1, and the graphics card used was NVIDIA 2080Ti with 12GB of video memory. In the data preparation module, the image was normalized to 120*400 before entering the CNN. Five convolution kernels with sizes of 16*16, 8*8, 4*4, 2*2, and 2*2 and five pooling kernels with sizes of 2*5, 2*5, 1*4, 1*2, and 1*2 were used in the CNN to perform feature extraction and dimensionality reduction on the image. The length, type, and number of channels of the convolution kernel can be adjusted. After CNN convolution, an output result with a size of 30*1*256 dimensions was obtained. The number of storage units in the Bi-LSTM was set to 256. In the experiment, batch_size was set to 32, ReLu was selected as the activation function, and the RMSPropOptimizer was used to optimize the loss in the network. These parts can be modified and adjusted. After the model training is completed, the character-accuracy and word-accuracy of letter recognition and word recognition are calculated respectively.

[0068] In summary, the embodiment of the present invention creates a recognition method for woodblock-printed Manchu characters. It combines Bi-LSTM, CTC, and CNN to extract and analyze more prominent and separable features from a large number of Manchu word images, thereby obtaining a highly accurate woodblock-printed Manchu recognition model.

[0069] The end-to-end neural network-based method for recognizing block-printed Manchu characters, proposed in this embodiment of the present invention, combines the advantages of a CNN (Convolutional Neural Network) in capturing more feature information with the Bi-LSTM (Bi-LSTM) method in linking contextual feature information. It identifies each Manchu word from the perspective of phonemes, balancing accuracy, stability, efficiency, and generalizability. The method achieved an accuracy rate of 86.98% for recognizing 20,000 images of block-printed Manchu characters.

[0070] Among existing Manchu recognition methods, traditional statistical methods segment and recognize Manchu strokes. However, these methods ignore the deformation caused by the combination of letters and only recognize specific Manchu letters, failing to recognize entire Manchu words. CNN-based recognition methods classify all Manchu words into 666 categories and recognize these words as a whole. However, in reality, the number of Manchu word types far exceeds 666, making these methods unable to accurately recognize Manchu words in categories that do not appear.

[0071] The comparison between current Manchu recognition methods is shown in Table 1.

[0072] Table 1 Comparison of different Manchu recognition methods

[0073]

[0074] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0075] From the above description of the embodiments, it can be seen that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0076] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without making any creative efforts.

[0077] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for recognizing Manchu characters printed on woodblock prints based on an end-to-end neural network, characterized in that: include: Cut out independent Manchu words from the pictures of ancient Manchu books, and represent each Manchu word with a Manchu word picture; Use convolutional neural network (CNN) to perform convolution operation on Manchu word images and extract the feature vectors of Manchu word images; Use the long short-term memory network (LSTM) to process the feature vectors extracted by CNN to obtain a vector sequence containing semantic information; The Connectivity Temporal Classification (CTC) algorithm is used to convert the vector sequence output by LSTM into a label sequence corresponding to Manchu words, and the recognition result of Manchu words is output based on the label sequence; The method of cutting out independent Manchu words from the picture of the ancient Manchu book and representing each Manchu word with a Manchu word picture includes: Extract individual Manchu words from each digitized page of an ancient Manchu book image, represent each Manchu word with a Manchu word image of varying length and width, scale all Manchu word images to the same size, normalize all word images to the same width while maintaining their aspect ratio, rotate each Manchu word image 90 degrees counterclockwise, assign distinct labels to all Manchu letters, and annotate all extracted woodblock printed Manchu words with glyph codes. The Connectivity Temporal Classification (CTC) algorithm is used to convert the vector sequence output by the LSTM into a label sequence corresponding to the Manchu word, and the Manchu word is recognized according to the label sequence, including: The CTC algorithm is used to convert the vector sequence output by the LSTM into a label sequence corresponding to the Manchu word. The conditional probabilities between different label sequences are calculated and compared, and the label sequence with the highest conditional probability value is output as the prediction result. The conditional probability of each letter in the Manchu word is defined as the probability of observing the corresponding letter image at a certain moment. The conditional probability of the entire Manchu word is generally obtained by summing the conditional probabilities after permuting different letters. The process of calculating the conditional probability of the label sequence by the CTC algorithm includes: the label sequence input to the CTC algorithm is represented by y1, y2, ..., y r Represented as T, where T is the length of the input sequence, L represents the 36 different character labels used, L' is a blank label added to L, that is, L'=L∪{blank}, where blank represents a blank label, π is defined as any possible label sequence, and there is The conditional probability of π is calculated as follows: in Indicates that label π appears at time t t The probability of p(π│y) in the sequence y1,y2,…,y r The conditional probability of the label sequence π appearing in ; exist A many-to-one mapping method B is defined above. The specific description is as follows: B first removes duplicate labels in π and removes all blank labels to obtain a new label sequence l. The conditional probability of l is defined as the sum of the conditional probabilities of the label sequence π mapped to l by B; The conditional probability calculation formula of l is as follows: In the no-dictionary mode, the sequence with the highest conditional probability in the label sequence is selected as the Manchu word recognition output. The calculation method is as follows: h(x)≈B(l′)#(5) Where h(x) is the final predicted word label sequence, and l′ represents the label sequence with the highest conditional probability; In the dictionary-based mode, the sequence that is closest to the word in the dictionary and has the highest conditional probability is selected as the recognition result output of the Manchu word; When the CNN is used to perform a convolution operation on the Manchu word image, the feature vector of the Manchu word image is extracted, including: CNN uses multiple sets of convolution kernels adapted to the feature size of Manchu words. The Manchu word image passes through multiple sets of convolution kernels of different sizes in the CNN. The length and width of the convolution kernel represent the size of the feature region extracted by each convolution operation. After each pass through the convolution kernel, the output feature map is reduced in dimensionality using max pooling. Max pooling selects the maximum value in the candidate region as the overall representation, ultimately obtaining a feature vector containing more word texture information. The method of using LSTM to process the feature vectors extracted by CNN to obtain a vector sequence includes: Use two-directional LSTM to process the feature vector of the Manchu word image and obtain the context information h1,h2,…,h contained in the feature vector T , all context information is integrated to obtain the vector sequence h containing context information output by Bi-LSTM i : in and Represents the backward and forward hidden state sequences respectively, and the final output vector sequence h i It is the sum of the two.

Images