A Chinese character recognition method based on holistic semantic feature extraction

Abstract

This invention discloses a Chinese character recognition method based on holistic semantic feature extraction, employing an Encoder-Decoder architecture. The method includes: preprocessing the input image; performing feature extraction and downsampling on the preprocessed input image using a convolutional neural network to obtain local feature maps containing different depth levels; feature fusion: extracting feature vectors at different levels from the local feature maps using Transformer Blocks of different depths, and concatenating the local feature maps and feature vectors at different levels along the channel dimension to form a concatenated feature vector; decoding: using a Transformer Decoder to perform autoregressive decoding on the concatenated feature vector to generate the recognition result. This method demonstrates high efficiency, accuracy, and adaptability in Chinese character recognition tasks, meeting diverse recognition needs and possessing high practical value and potential for widespread application.

Description

Technical Field

[0001] This invention relates to the field of character detection and recognition, and in particular to a Chinese character recognition method based on the extraction of holistic semantic features. Background Technology

[0002] Optical Character Recognition (OCR) is the process of converting scanned text images or handwritten text into editable text. It is of great significance in the fields of computer vision and natural language processing. OCR technology originated in the 1960s and has a history of several decades. During this period, marked by the rise of deep learning, the development of OCR can be divided into the traditional stage and the deep learning stage.

[0003] Traditional OCR technology includes steps such as image preprocessing (processing the input image, such as denoising, enhancement, binarization, etc., to improve image quality), character segmentation (segmenting text in the image into individual characters), feature extraction (extracting character features for recognition), character recognition (using pattern recognition algorithms to classify and recognize characters), and post-processing (correcting and optimizing the recognition results). Traditional OCR technology relies on image quality and performs poorly when processing blurry, tilted, unevenly lit, or complex background images. Furthermore, it requires expert knowledge and experience to design and select effective features, increasing system complexity and development difficulty. With the introduction of deep learning, OCR technology allows models to automatically learn complex features from raw data, eliminating the need for manually designed feature extraction algorithms. This direct mapping from image pixels to text characters simplifies the OCR process and enables end-to-end learning. Deep learning models, due to their powerful feature learning capabilities, can better handle low-quality and complex images (blurry, tilted, unevenly lit), adapting to a wider range of practical applications.

[0004] Existing deep learning OCR technologies mainly include Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). CNNs play a crucial role in Optical Character Recognition (OCR) technology, primarily responsible for feature extraction of text in images. Through multiple layers of convolution and pooling operations, CNNs can automatically learn text patterns in images, such as letter shapes and edge information. This ability to automatically learn features reduces reliance on traditional image processing techniques, such as edge detection and feature engineering. Furthermore, the translation invariance of CNNs makes the model insensitive to changes in text position, improving recognition accuracy. On English datasets, due to the limited number of characters and simple structure, CNNs can efficiently extract key features helpful for recognition, thus achieving high-accuracy character recognition. Recurrent Neural Networks (RNNs) and their variants, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), are used in OCR technology to process sequential data, especially in recognizing the character order in words or sentences. RNNs can capture long-term dependencies in time-series data, which is crucial for understanding text content and context. In OCR, RNNs transform the static features extracted by CNNs into dynamic sequence representations, predicting entire words or sentences by learning the relationships between characters. Furthermore, the combination of RNNs with the CTC loss function allows the model to be trained without explicit sequence alignment, effectively handling the problem of variable-length input and output sequences. In English OCR tasks, RNNs can effectively convert recognized characters in images into correct text sequences, significantly improving the accuracy and coherence of recognition.

[0005] Existing deep learning OCR technologies, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), achieve high accuracy on English and Latin text scene image datasets, but their accuracy needs improvement on Chinese scene image datasets. The main reason is that Chinese scene image recognition presents the following challenges:

[0006] Differences in character structure: English characters (letters) are limited in number (26) and have a simple structure, while Chinese characters are numerous, with thousands of commonly used characters, and have complex structures with diverse combinations of strokes and radicals.

[0007] Language model differences: English words are separated by spaces, while Chinese text does not have obvious word separators, which makes Chinese language models and text segmentation more complex.

[0008] Training data distribution: The performance of a deep learning model depends heavily on the quality and distribution of its training data. If a model is primarily trained on English datasets, it may not generalize well to Chinese datasets.

[0009] Contextual information: Contextual information in Chinese text is crucial for recognition because many Chinese characters have different meanings in different contexts. English models may not learn this kind of contextual information.

[0010] Annotation and corpus resources: High-quality Chinese annotated datasets are relatively scarce compared to English datasets, which limits model training and optimization.

[0011] These challenges result in traditional deep learning OCR techniques performing poorly when transferred to Chinese datasets. Summary of the Invention

[0012] In order to overcome the above-mentioned shortcomings and deficiencies of the existing technology, the purpose of this invention is to provide a Chinese character recognition method based on the extraction of holistic semantic features.

[0013] This method improves the deep learning architecture by combining Convolutional Neural Networks (CNNs) and Transformers. The algorithm integrates the strong feature extraction capabilities of CNNs with the long sequence processing capabilities of Transformers, making it particularly suitable for character recognition tasks in Chinese contexts. By combining local and global attention mechanisms, the model can more accurately capture the details and contextual information of Chinese characters, thereby improving recognition accuracy.

[0014] The objective of this invention is achieved through the following technical solution:

[0015] A Chinese character recognition method based on holistic semantic feature extraction, employing an Encoder-Decoder architecture, includes:

[0016] Image acquisition: Preprocessing the input image;

[0017] Feature extraction: The preprocessed input image is subjected to feature extraction and downsampling through a convolutional neural network to obtain local feature maps containing different depth levels in the input image. The convolutional neural network includes CNN convolutional layers and MaxPool layers.

[0018] Feature fusion: The feature vectors of different levels in the local feature map are extracted using Transformer Blocks of different depths. The local feature map and the feature vectors of different levels extracted by the Transformer Block are concatenated in the channel dimension to form the concatenated feature vector.

[0019] Decoding: Use Transformer Decoder to perform autoregressive decoding on the concatenated feature vectors to generate recognition results.

[0020] Furthermore, the feature fusion step includes three Transformer Blocks, each of which includes a multi-head attention mechanism, a local attention mechanism, and a downsample operation.

[0021] Furthermore, the three Transformer Blocks are the first Transformer Block, the second Transformer Block, and the third Transformer Block, respectively.

[0022] The local attention mechanism in Transformer Block limits the calculation of attention to a local window centered on the corresponding character;

[0023] The Downsample operation in the Transformer Block concatenates two adjacent horizontally arranged vectors along the channel dimension.

[0024] Downsample is disabled in the first TransformerBlock, but enabled in the second and third TransformerBlocks.

[0025] Furthermore, the feature fusion specifically includes:

[0026] The local feature map extracts features through the first Transformer Block to obtain the first feature vector;

[0027] The first feature vector is input into the second Transformer Block to extract features and obtain the second feature vector.

[0028] The second feature vector is input into the third Transformer Block to extract features and obtain the third feature vector.

[0029] The local feature map, the first feature vector, and the second feature vector are linearly transformed through a fully connected layer and then concatenated to form the fourth feature vector.

[0030] The third and fourth feature vectors are concatenated and then input into the Transformer Decoder for decoding.

[0031] Furthermore, the number of layers in the Transformer Decoder is set to 1.

[0032] Furthermore, the performance of the text recognition method is evaluated using accuracy and normalized edit distance.

[0033] Furthermore, the CNN convolutional layer includes multiple 3×3 convolutional modules.

[0034] Furthermore, the local and first and second feature vectors are transformed into feature vectors of a unified dimension through a linear fully connected layer, while the fourth feature vector does not undergo dimension transformation through a linear fully connected layer.

[0035] Furthermore, during training, the Transformer Decoder uses complete text sequences as input and employs masked attention to prevent "cheating."

[0036] During prediction, a greedy search or beam search strategy is selected based on the probability distribution of generating the next character from the preceding text.

[0037] Furthermore, the recognition result is in text form.

[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0039] This method employs the Transformer Encoder-Decoder architecture, which is highly effective in processing sequence data, especially in Natural Language Processing (NLP) tasks. Applying it to OCR tasks, particularly Chinese character recognition, allows for better capture of dependencies between characters.

[0040] This method extracts features at both depth and width through convolutional and pooling layers, and then further processes the features using Transformer Blocks. This approach, combining CNN and Transformer, can extract rich features at different levels, enhancing the model's understanding of the details and structure of Chinese characters.

[0041] This method employs a local and multi-head attention mechanism: allowing the model to simultaneously focus on local stroke details and the overall layout of the Chinese character. This mechanism helps the model better distinguish between Chinese characters with similar shapes, thus improving recognition accuracy.

[0042] This method extracts and fuses feature maps from different depths of the Encoder, which helps to preserve the details of Chinese characters, such as radicals or small strokes. This information is crucial for accurate Chinese character recognition. Attached Figure Description

[0043] Figure 1 This is a flowchart of the process of the present invention. Detailed Implementation

[0044] The present invention will be further described in detail below with reference to the embodiments, but the implementation of the present invention is not limited thereto.

[0045] Example

[0046] like Figure 1 As shown, a Chinese character recognition method based on holistic semantic feature extraction adopts an Encoder-Decoder architecture, including:

[0047] The image is acquired and preprocessed. The Chinese image is adjusted to the size required by the Encoder model to ensure image consistency. In this embodiment, the input image is usually adjusted to 32 pixels high and 256 pixels wide with 3 channels. The "start" identifier is used as input in the Decoder part. After that, the model will output the recognition result from left to right in an autoregressive manner.

[0048] Feature extraction: The preprocessed input image is subjected to feature extraction and downsampling through a convolutional neural network to obtain local feature maps of different depth levels in the input image. The convolutional neural network includes CNN convolutional layers and MaxPool layers. The CNN convolutional layer includes multiple 3×3 convolutional modules.

[0049] A series of 3×3 convolution modules are used to perform convolution operations on the image to extract local features. The convolution operation identifies local patterns in the image through a series of filters.

[0050] Then, a max pooling layer is used to downsample the convolutional feature map to reduce the spatial resolution and size of the feature map while preserving the feature information.

[0051] Feature fusion utilizes Transformer Blocks of different depths to extract feature vectors at different levels from the local feature map. The local feature map and the feature vectors extracted from the Transformer Block at different levels are then concatenated along the channel dimension to form a concatenated feature vector.

[0052] The Transformer Blocks at different depths include three Transformer Blocks: the first Transformer Block, the second Transformer Block, and the third Transformer Block.

[0053] Each Transformer Block includes a multi-head attention mechanism, a local attention mechanism, and a downsample operation.

[0054] The local attention mechanism in the Transformer Block limits the computation of attention to a local window centered on the corresponding character. By limiting the scope of the attention mechanism, the model can pay more attention to the local strokes and structure of Chinese characters, which helps to distinguish Chinese characters with similar shapes.

[0055] Multi-head attention mechanism: allows the model to consider all elements when processing the entire sequence, which helps to capture the global layout and arrangement rules of Chinese characters.

[0056] Downsample operation: concatenates adjacent vectors along the channel dimension to reduce the width of the feature map while maintaining its height, in order to accommodate Chinese characters with different arrangements.

[0057] The Downsample operation in the Transformer Block concatenates two adjacent horizontally arranged vectors along the channel dimension.

[0058] Downsample is disabled in the first TransformerBlock, but enabled in the second and third TransformerBlocks.

[0059] The specific process of feature fusion is as follows:

[0060] The local feature map extracts features through the first Transformer Block to obtain the first feature vector;

[0061] The first feature vector is input into the second Transformer Block to extract features and obtain the second feature vector.

[0062] The second feature vector is input into the third Transformer Block to extract features and obtain the third feature vector.

[0063] The local feature map, the first feature vector, and the second feature vector are linearly transformed through a fully connected layer and then concatenated to form the fourth feature vector.

[0064] The third and fourth feature vectors are concatenated and then input into the Transformer Decoder for decoding.

[0065] This Chinese character recognition model architecture achieves overall semantic feature extraction, while local feature maps extract shallow feature representations. The first feature vector further extracts features, the second feature vector further extracts features from the first feature vector, and the third feature vector further extracts deep-level feature representations. Concatenating feature vectors from different levels yields a feature vector that encompasses the overall semantic meaning.

[0066] Decoding: Use Transformer Decoder to perform autoregressive decoding on the concatenated feature vectors to generate recognition results.

[0067] Specifically, the Decoder adopts a single-layer Transformer structure, taking the encoder's output as input. The Key and Value using cross attention operations generate recognition results from left to right through autoregression.

[0068] During training, the model of this invention uses complete text sequences to feed into the Decoder and employs masked attention to prevent "cheating".

[0069] When making predictions, based on the probability distribution of generating the next character from the preceding text, either a greedy search or a beamsearch strategy can be chosen.

[0070] Greedy search: Selects the character with the highest probability as the next character, which is efficient but may get stuck in local optima. Beam search: Considers the k sequences with the highest scores, which is more comprehensive but consumes more computational resources. The score calculation for beam search is shown in equation (1):

[0071]

[0072] Where x is the Encoder input, which can be the source language text in an NLP translation task or an input image. y1, y2…y n-1 For each character at a position, p is the probability and S is the path score. Since the longer the path, the lower the score will be, in order to avoid a more suitable long output text being eliminated by a short output text, a penalty term based on the text length is usually added to S, as shown in Equation (2), where length_penalty is a custom parameter.

[0073]

[0074] When using a greedy search, the model stops generating when it generates an end marker or reaches the maximum generation length. When using beam search, the model stops generating when it generates an end marker for the current highest-scoring sequence or reaches the maximum generation length.

[0075] The evaluation metrics used in this method are accuracy (ACC) and normalized edit distance (NED) to assess model performance.

[0076] The input image is scaled to the same size, and then each pixel value in the image is divided by 255 to normalize it to the 0-1 range. The resulting value is then subtracted from the mean and divided by the standard deviation to standardize it. The normalized image is then passed through a CNN convolutional layer and a max pooling layer to extract feature maps.

[0077] In the described model, convolutional layers (CNN) and max-pooling layers (MaxPool) together constitute the Encoder part, used to efficiently extract features from the input Chinese scene images. CNN identifies local patterns in the image, such as edges, textures, and shapes, through local receptive fields and learnable convolutional kernels. These local features are crucial for distinguishing different strokes of Chinese characters. Simultaneously, the parameter sharing and translation invariance of CNNs make the model insensitive to small positional changes in characters within the image, enhancing its generalization ability. Next, the max-pooling operation further abstracts features by reducing the spatial resolution of the feature map. It preserves the most salient visual features by selecting the maximum value within a local region, while reducing the computational burden on the model. This downsampling process not only simplifies the model but also increases the network's receptive field, enabling the model to capture more global features, such as the structure and layout of the entire Chinese character. Through this feature extraction strategy combining CNN and MaxPool, this method can construct multi-level feature representations from low to high levels, providing rich visual information for subsequent TransformerBlock processing. These features are crucial for accurately recognizing Chinese characters in images because they contain both local details and global structure of the characters. Overall, the collaborative work of the CNN convolutional layers and MaxPool provides a solid feature extraction foundation for the entire character recognition model, enabling it to efficiently process complex Chinese scene images and accurately recognize the text information within them.

[0078] The Transformer Block plays a crucial role in the described model, responsible for in-depth processing and fusion of image features. Each Transformer Block, through a self-attention mechanism, allows the model to simultaneously focus on all elements in the sequence. This significantly improves the model's ability to capture global dependencies in the image compared to traditional convolutional operations that rely solely on local information. In particular, the multi-head attention design enables the model to process information from multiple subspaces in parallel, enhancing the recognition and integration of different features. Local attention focuses on capturing the local strokes and structure of Chinese characters. This design allows the model to distinguish similar-shaped characters when handling complex Chinese character recognition tasks, while also understanding the contextual relationships of characters within the image. Furthermore, the feed-forward neural network (FFNN) in the Transformer Block further performs non-linear transformations on the output of self-attention, enhancing the model's expressive power. The application of residual connections and layer normalization ensures information flow and training stability within the deep network. Through these structures, Transformer Blocks can effectively extract and fuse image features, providing the model with rich information to generate accurate recognition results. In Chinese character recognition, this fine-grained processing of local and global features is key to improving recognition accuracy and robustness.

[0079] The combination of Transformer Block and CNN offers significant advantages for Chinese scene character recognition tasks in deep learning models. The CNN part leverages its powerful local feature extraction capabilities, using convolutional and pooling layers to identify basic visual elements in images such as edges, textures, and shapes. These local features are crucial for capturing the basic strokes and structure of Chinese characters. Subsequently, these features are fed into the Transformer Block, which further processes them through a self-attention mechanism, enabling the model to capture the relationships between strokes within Chinese characters and the interactions between characters.

[0080] The multi-head attention mechanism in the Transformer Block allows the model to process information in parallel across different subspaces, enhancing its understanding of image feature diversity and improving its ability to integrate local and global features of Chinese characters. Furthermore, the application of residual connections and layer normalization ensures efficient information flow within deep networks, stabilizes the training process, and helps avoid gradient vanishing or exploding problems.

[0081] This combination enables the model not only to recognize local details in images but also to understand how these details combine into meaningful wholes within a larger context. This is particularly important for Chinese character recognition, given the complex structure of Chinese characters and the large number of characters with similar shapes. Furthermore, this structure's advantage lies in its ability to capture long-distance dependencies, which is crucial for recognizing the order and contextual relationships of Chinese characters in images. Overall, the combination of CNN and Transformer Block provides a powerful feature extraction and sequence modeling framework for Chinese character recognition tasks, significantly improving the model's accuracy and robustness.

[0082] In the described method, the Transformer Decoder plays a crucial role in converting the image features extracted by the Encoder into the final text output. This process leverages a self-attention mechanism, allowing the model to consider all previously generated characters in the sequence when generating each character, thereby capturing long-distance dependencies, which is essential for understanding and generating coherent text sequences. The autoregressive nature of the Decoder means that the generation of each character depends on the previous character, mimicking the natural process of human reading and writing. Structurally, the Decoder consists of multiple identical layers, each including a self-attention layer, an encoder-decoder attention layer, and a feedforward neural network. The self-attention layer allows the model to focus on the already generated output, while the encoder-decoder attention layer allows the model to focus on the features provided by the Encoder. Furthermore, the application of residual connections and layer normalization ensures the efficient flow of information in the deep network and helps stabilize the training process. Regarding the generation strategy, the Decoder can employ either greedy search or beam search. Greedy search is simple and efficient, generating sequences by selecting the character with the highest probability; while beam search finds the optimal character sequence by considering multiple candidate characters and their combinations. This flexibility allows the model to balance computational efficiency and recognition accuracy according to actual needs.

[0083] In the described method, the loss function is a crucial part of the training process used to measure the difference between the model's predictions and the actual annotations, guiding the optimization direction of the model parameters. The algorithm employs the Cross-Entropy Loss function, a loss function commonly used in classification tasks, especially in natural language processing and text recognition.

[0084] The cross-entropy loss function measures the difference between the probability distribution of the model's output and the probability distribution of the true labels. In text recognition tasks, the model's output is typically a probability distribution where each position represents a possible character or symbol, while the true labels are a one-hot encoded vector where the correct character position is 1 and the rest are 0.

[0085] The loss function can be calculated as follows:

[0086] L=-∑iyilog(pi) (3)

[0087] Where yi is the one-hot encoded vector of the true label, pi is the probability distribution predicted by the model, and L is the loss value. The cross-entropy loss function quantifies the difference between the true label and the model prediction into a numerical value using the negative log-likelihood. During training, the goal is to minimize the value of the loss function by adjusting the model parameters.

[0088] Furthermore, since the score decreases with longer paths, a text length-based penalty term is typically added to the loss function to prevent a more suitable long output text from being rejected by a short output text. This penalty term is a custom parameter used to balance the impact of text length on the loss function.

[0089] In actual training, the model's parameters are updated using the backpropagation algorithm, and gradient descent or its variants (such as the Adam optimizer) are used to minimize the loss function. In this way, the model learns how to accurately map the features of the input image to the correct character sequences, thereby improving the accuracy of character recognition.

[0090] In the described algorithm, evaluation metrics are important tools for measuring model performance, providing a quantitative standard for evaluating the model's quality. This algorithm employs two main evaluation metrics: accuracy (ACC) and normalized edit distance (NED).

[0091] Accuracy (ACC): Accuracy is a straightforward evaluation metric used to measure the proportion of characters correctly recognized by the model out of the total number of characters. In text recognition tasks, accuracy directly reflects the model's recognition ability.

[0092]

[0093] Here, N is the total number of images in the test set, and δ is an indicator function that returns 1 when the predicted result equals the true label, and 0 otherwise. High accuracy means that the model can correctly identify characters in the image most of the time.

[0094] Normalized Edit Distance (NED): Normalized edit distance is a more granular evaluation metric that measures the difference between the model output and the ground truth annotations. Edit distance refers to the minimum number of single-character edits (insertion, deletion, or replacement) required to transform one string into another. Normalized edit distance is normalized by dividing by the maximum character length of the input image, ensuring the result is between 0 and 1. The formula is:

[0095]

[0096] Where distance represents the edit distance between the predicted result and the ground truth label, and maxlen is the maximum character length across all images. The lower the NED value, the smaller the difference between the model's prediction and the ground truth label, meaning the better the model's performance.

[0097] Dataset type and source used during training

[0098] Scene datasets: These datasets typically contain images of Chinese text taken in various scenes, such as street scenes, billboards, and signs. These images reflect the diversity and complexity of Chinese characters in the real world.

[0099] Web datasets may contain images of Chinese text scraped from the internet, including webpage screenshots, online documents, etc., reflecting the characteristics of text in the online environment.

[0100] Document datasets: These datasets may consist of scanned or photographed document pages containing formatted text layouts, such as books, articles, reports, etc.

[0101] Dataset processing and use

[0102] Before using the dataset, the documentation mentions that the images were resized to a uniform height of 32 pixels and width of 256 pixels, which helps the model process images from different sources in a consistent manner.

[0103] The model's performance on different datasets is analyzed, and the algorithm's performance is compared with other existing algorithms (such as CRNN, ASTER, MORAN, SAR, SEED, and TransOCR).

[0104] Fine-tuning strategy

[0105] This method proposes a fine-tuning strategy: first train the model on the Scene dataset, and then fine-tune it on a dataset specific to the application scenario (such as storefront text recognition). This strategy helps the model better adapt to specific recognition tasks and improves recognition accuracy.

[0106] ablation experiment

[0107] Ablation studies are a common experimental method in machine learning and deep learning research, used to evaluate the contribution and importance of various components in a model. This method allows researchers to understand which parts of the model significantly impact the final performance and which parts may be unnecessary. This model underwent ablation studies on the Scene, Web, and Document datasets, with deep branches and local attention removed respectively. The figure shows the three different experimental settings and their performance on different datasets:

[0108] w / o deep branch: Indicates the model version that does not use deep learning branches. Performance on the Web, Document, and Scene datasets is 64.12%, 67.31%, and 95.77%, respectively.

[0109] w / o local attention: Indicates the model version without using the local attention mechanism. Performance on the three datasets is 62.64%, 65.72%, and 95.53%, respectively.

[0110] Ours: Represents the complete model, including all features and techniques. Performance on the three datasets is 65.98%, 68.41%, and 96.60%, respectively.

[0111] The ablation experiment demonstrates that the "Ours" model outperforms models that remove deep branches and local attention on Document, Scene, and Web datasets.

[0112] Based on this data, as shown in Table 1, we can conclude that deep learning branches and local attention mechanisms are beneficial for improving model performance on specific datasets. The complete model (Ours) exhibits the best performance across all datasets, demonstrating that the synergistic effect of the various components within the model is effective in improving overall recognition accuracy.

[0113] Table 1

[0114]

[0115] Comparative experiment

[0116] Comparative experiments are a method to quantify the performance of different models or algorithms by running and evaluating them under identical conditions. They help researchers understand the strengths and limitations of each model, enabling them to select the most suitable technology for specific application scenarios. In the fields of machine learning and deep learning, comparative experiments typically involve multiple metrics such as accuracy, processing speed, and generalization ability to ensure a comprehensive evaluation of the model's performance in real-world applications.

[0117] As shown in Table 2, this comparative experiment involved CRNN, ASTER, MORAN, SAR, SEED, and TransOCR models, conducted on the Scene, Web, and Document datasets. The model achieved the best performance on the Web dataset, with a performance of 68.30 / 0.822. It also achieved good performance on the other two datasets, with performances of 65.99 / 0.802 and 96.60 / 0.995, respectively. On the Scene dataset, its performance surpassed that of CRNN, ASTER, MORAN, SAT, and SEED models. On the Document dataset, its performance surpassed that of MORAN, SAR, and SEED datasets. This comparative experiment demonstrates that our model has performance advantages in handling text recognition tasks in different scenarios. Our model exhibits high accuracy and low edit distance, meaning that our model has high accuracy and reliability in recognizing text on these datasets. However, it should be noted that the statement "surpasses" is relative to other models in the provided datasets and does not necessarily mean that it is the best performing model among all possible models. Furthermore, other factors need to be considered in practical applications, such as model complexity, training time, and resource consumption.

[0118] Table 2

[0119]

[0120] This algorithm demonstrates significant advantages in Chinese character recognition tasks, primarily due to its Transformer architecture, which is particularly well-suited for processing sequential data and provides robust technical support for Chinese character recognition. By combining local and global attention mechanisms, the algorithm accurately captures the local strokes and overall layout of Chinese characters, greatly improving the recognition accuracy for similar-looking characters. Furthermore, the application of feature fusion technology allows the model to retain more details of Chinese characters, such as radicals or small strokes, further enhancing recognition accuracy. The optimized design of the downsample operation ensures that key Chinese character information is not lost while reducing the feature map size, especially for vertically or diagonally arranged characters, avoiding adverse effects. The provision of autoregressive output and two generation strategies—greedy search and beam search—achieves a balance between accuracy and computational efficiency, adapting to different application needs. The comprehensive application of evaluation metrics ACC and NED provides a quantitative assessment of model performance. The introduction of fine-tuning strategies allows the model to quickly adapt to new application scenarios, enhancing its practicality. These advantages comprehensively demonstrate the efficiency, accuracy, and adaptability of this algorithm in Chinese character recognition tasks, meeting diverse recognition needs and possessing high practical value and promotion potential.

[0121] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A Chinese character recognition method based on holistic semantic feature extraction, characterized in that, It adopts an Encoder-Decoder architecture, including: Image acquisition: Preprocessing the input image; Feature extraction: The preprocessed input image is subjected to feature extraction and downsampling through a convolutional neural network to obtain local feature maps containing different depth levels in the input image. The convolutional neural network includes CNN convolutional layers and MaxPool layers. Feature fusion: The feature vectors of different levels in the local feature map are extracted using Transformer Blocks of different depths. The local feature map and the feature vectors of different levels extracted by the Transformer Block are concatenated in the channel dimension to form the concatenated feature vector. Decoding: The Transformer Decoder is used to perform autoregressive decoding on the concatenated feature vectors to generate recognition results; The feature fusion step includes three Transformer Blocks, each of which includes a multi-head attention mechanism, a local attention mechanism, and a downsample operation; The three Transformer Blocks are the first Transformer Block, the second Transformer Block, and the third Transformer Block, respectively. The feature fusion specifically refers to: The local feature map extracts features through the first Transformer Block to obtain the first feature vector; The first feature vector is input into the second Transformer Block to extract features and obtain the second feature vector. The second feature vector is input into the third Transformer Block to extract features and obtain the third feature vector. The local feature map, the first feature vector, and the second feature vector are linearly transformed through a fully connected layer and then concatenated to form the fourth feature vector. The third and fourth feature vectors are concatenated and then input into the Transformer Decoder for decoding.

2. The Chinese character recognition method according to claim 1, characterized in that, The local attention mechanism in Transformer Block limits the calculation of attention to a local window centered on the corresponding character; The Downsample operation in Transformer Block concatenates two adjacent horizontally arranged vectors along the channel dimension. Downsample is disabled in the first TransformerBlock, but enabled in the second and third TransformerBlocks.

3. The Chinese character recognition method according to claim 1, characterized in that, Set the number of layers in the Transformer Decoder to 1.

4. The Chinese character recognition method according to any one of claims 1-3, characterized in that, The performance of the text recognition method was evaluated using accuracy and normalized edit distance.

5. The Chinese character recognition method according to claim 1, characterized in that, The CNN convolutional layer includes multiple 3×3 convolutional modules.

6. The Chinese character recognition method according to claim 1, characterized in that, The local feature map, the first feature vector, and the second feature vector are converted into feature vectors of a unified dimension through a linear fully connected layer, while the fourth feature vector is not converted into a dimension through a linear fully connected layer.

7. The Chinese character recognition method according to claim 1, characterized in that, During training, the Transformer Decoder is fed with complete sequences of text and employs masked attention to prevent "cheating". During prediction, a greedy search or beam search strategy is selected based on the probability distribution of generating the next character from the preceding text.

8. The Chinese character recognition method according to claim 1, characterized in that, The recognition result is in text format.

Images