Chinese character recognition method based on overall semantic feature extraction

Abstract

The invention discloses a Chinese character recognition method based on overall semantic feature extraction, and the method employs an Encoder-Decoder architecture, and comprises the steps: carrying out the preprocessing of an input image; performing feature extraction and down-sampling on the preprocessed input image through a convolutional neural network to obtain local feature maps of different depth levels of the input image; feature fusion: extracting feature vectors of different levels in the local feature map by using Transform Blocks of different depths, and splicing the local feature map and the feature vectors of different levels on a channel dimension to form spliced feature vectors; and decoding: carrying out autoregressive decoding on the spliced feature vectors by using Transform Decoder, and generating an identification result. The method has high efficiency, accuracy and adaptability in a Chinese character recognition task, meets diversified recognition requirements, and has very high practical value and popularization potential.

Description

Technical Field

[0001] The present invention relates to the field of text detection and recognition, and in particular to a Chinese text recognition method based on overall semantic feature extraction. Background Art

[0002] Optical Character Recognition (OCR) is the process of converting scanned text images or handwritten text into editable text. It is of great significance in the fields of computer vision and natural language processing. Text recognition technology originated in the 1960s, and it has a history of several decades. During this period, with the rise of deep learning as a node, the development of OCR can be divided into the traditional stage and the deep learning stage.

[0003] Traditional OCR technology includes image preprocessing (processing the input image, such as denoising, enhancement, binarization, etc., to improve image quality), character segmentation (segmenting the text in the image into individual characters), feature extraction (extracting the features of the characters for recognition), character recognition (using pattern recognition algorithms to classify and recognize characters), and post-processing (correcting and optimizing the recognition results). Traditional OCR technology relies on image quality and is not effective when processing blurred, tilted, unevenly illuminated or complex background images. It also requires expert knowledge and experience to design and select effective features, which increases the complexity and development difficulty of the system. After the introduction of deep learning into OCR technology, the model can automatically learn complex features from raw data without the need to manually design feature extraction algorithms, that is, directly mapping from image pixels to text characters, which simplifies the OCR process and realizes end-to-end learning. Due to its powerful feature learning ability, the deep learning model can better handle low-quality images and complex images (blurred, tilted, unevenly illuminated), and can adapt to more practical application scenarios.

[0004] Existing deep learning OCR technologies mainly include convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Convolutional neural networks (CNNs) play a vital role in optical character recognition (OCR) technology, and are mainly responsible for feature extraction of text in images. Through multiple layers of convolution and pooling operations, CNNs can automatically learn text patterns in images, such as the shape and edge information of letters. This ability to automatically learn features reduces the reliance on traditional image processing techniques, such as edge detection and feature engineering. In addition, the translation invariance of CNNs makes the model insensitive to changes in the position of text, improving recognition accuracy. On English datasets, due to the limited number of characters and simple structure, CNNs can efficiently extract key features that are helpful for recognition, thereby achieving high-accuracy character recognition. Recurrent neural networks (RNNs) and their variants, such as long short-term memory networks (LSTMs) and gated recurrent units (GRUs), are used in OCR technology to process sequence data, especially in recognizing the order of characters in words or sentences. RNNs can capture long-term dependencies in time series data, which is crucial for understanding text content and context. In OCR, RNN converts the static features extracted by CNN into dynamic sequence representations, and predicts the entire word or sentence by learning the relationship between characters. In addition, the combination of RNN and CTC loss function allows the model to be trained without explicit sequence alignment, effectively handling the problem of variable-length input and output sequences. In the English OCR task, RNN can effectively convert the recognized characters in the image into the correct text sequence, significantly improving the recognition accuracy and coherence.

[0005] Existing deep learning OCR technologies, such as convolutional neural networks (CNN) and recurrent neural networks (RNN), have achieved high accuracy on English and Latin text scene image datasets, but the accuracy on Chinese scene image datasets needs to be improved. The main reason is that Chinese scene image recognition has the following difficulties:

[0006] Differences in character structure: The number of English characters (letters) is limited (26) and the structure is simple, while the number of Chinese characters is huge. There are thousands of commonly used Chinese characters with complex structures and various combinations of strokes and radicals.

[0007] Language model differences: English words are separated by spaces, while Chinese text has no obvious word separators, which makes language modeling and text segmentation more complicated for Chinese.

[0008] Training data distribution: The performance of deep learning models depends heavily on the quality and distribution of the training data. If the model is trained primarily on English datasets, it may not generalize well to Chinese datasets.

[0009] Contextual information: The contextual information of Chinese text is very important for recognition, because many Chinese characters have different meanings in different contexts. English models may not learn this contextual information.

[0010] Annotation and corpus resources: High-quality Chinese annotated datasets may be relatively scarce compared to English, which limits the training and optimization of the model.

[0011] These difficulties result in poor performance of traditional deep learning OCR technology when transferred to Chinese datasets. Summary of the invention

[0012] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, an object of the present invention is to provide a Chinese character recognition method based on overall semantic feature extraction.

[0013] This method improves the deep learning architecture and combines the convolutional neural network (CNN) and the Transformer architecture. The entire algorithm combines the strong feature extraction capability of CNN and the long sequence processing capability of Transformer, which is particularly suitable for processing text recognition tasks in Chinese scenarios. Through the combination of local and global attention mechanisms, the model can more accurately capture the details and contextual information of Chinese characters, thereby improving recognition accuracy.

[0014] The purpose of the present invention is achieved through the following technical solutions:

[0015] A Chinese character recognition method based on overall semantic feature extraction adopts an Encoder-Decoder architecture, including:

[0016] Get image: preprocess the input image;

[0017] Feature extraction: The preprocessed input image is subjected to feature extraction and downsampling through a convolutional neural network to obtain local feature maps of the input image at different depth levels. The convolutional neural network includes a CNN convolutional layer and a MaxPool layer.

[0018] Feature fusion: Use Transformer Blocks of different depths to extract feature vectors of different levels in the local feature map, and concatenate the local feature map and feature vectors of different levels extracted by Transformer Block in the channel dimension to form a concatenated feature vector;

[0019] Decoding: Use Transformer Decoder to perform autoregressive decoding on the concatenated feature vector to generate the recognition result.

[0020] Furthermore, the feature fusion step includes three Transformer Blocks, each of which includes a multi-head attention mechanism, a local attention mechanism and a Downsample operation.

[0021] Further, the three Transformer Blocks are respectively a first Transformer Block, a second Transformer Block and a third Transformer Block;

[0022] The local attention mechanism in Transformer Block limits the attention calculation to a local window centered on the corresponding character;

[0023] The Downsample operation in the Transformer Block concatenates two adjacent horizontally arranged vectors in the channel dimension.

[0024] The Downsample operation is disabled in the first TransformerBlock, and is enabled in the second and third TransformerBlocks.

[0025] Furthermore, the feature fusion is specifically as follows:

[0026] The local feature map is extracted through the first Transformer Block to obtain the first feature vector;

[0027] The first feature vector is input into the second Transformer Block to extract features and obtain the second feature vector;

[0028] The second feature vector is input into the third Transformer Block to extract features and obtain the third feature vector;

[0029] The local feature map, the first feature vector and the second feature vector are linearly transformed through a fully connected layer and then concatenated into a fourth feature vector;

[0030] After the third eigenvector and the fourth eigenvector are concatenated, they are input into the Transformer Decoder for decoding.

[0031] Furthermore, the number of layers of Transformer Decoder is set to 1.

[0032] Furthermore, the performance of the text recognition method is evaluated using accuracy and normalized edit distance.

[0033] Furthermore, the CNN convolutional layer includes multiple 3×3 convolutional modules.

[0034] Furthermore, the local and first and second eigenvectors are converted into eigenvectors of uniform dimension through linear full connection, and the fourth eigenvector is not converted in dimension through the linear full connection layer.

[0035] Furthermore, when training Transformer Decoder, the complete text sequence is fed into the Decoder, and masked attention is used to prevent "cheating";

[0036] When predicting, choose a greedy search or beam search strategy based on the probability distribution of the next character generated by the previous text.

[0037] Furthermore, the recognition result is in text form.

[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0039] This method uses the Transformer Encoder-Decoder architecture, which is very effective in processing sequence data, especially in natural language processing (NLP) tasks. Applying it to OCR tasks, especially in Chinese character recognition, can better capture the dependencies between characters.

[0040] The depth and width of feature extraction in this method are extracted through convolutional layers and pooling layers, and then further processed using Transformer Block. This method that combines CNN and Transformer can extract rich features at different levels and enhance the model's understanding of the details and structure of Chinese characters.

[0041] This method uses a local and multi-head attention mechanism: allowing the model to focus on both local stroke details and the overall layout of Chinese characters. This mechanism helps the model better distinguish Chinese characters with similar shapes and improves recognition accuracy.

[0042] The feature fusion of this method extracts feature maps from different depths of the Encoder and fuses them, which helps to retain the details of Chinese characters, such as radicals or small strokes. This information is crucial for accurate recognition of Chinese characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a work flow chart of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below in conjunction with examples, but the embodiments of the present invention are not limited thereto.

[0045] Example

[0046] like Figure 1 As shown, a Chinese character recognition method based on overall semantic feature extraction adopts an Encoder-Decoder architecture, including:

[0047] Get the image and preprocess the input image. The input Chinese picture will be adjusted to the size required by the Encoder model to ensure the consistency of the image. In this embodiment, the input image is usually adjusted to 32 pixels in height, 256 pixels in width, and 3 channels. The "start" identifier is used as input in the Decoder part. After that, the model will output the recognition results from left to right in an autoregressive manner.

[0048] Feature extraction: The preprocessed input image is subjected to feature extraction and downsampling through a convolutional neural network to obtain a local feature map of the input image containing different depth levels. The convolutional neural network includes a CNN convolutional layer and a MaxPool layer. The CNN convolutional layer includes multiple 3×3 convolutional modules.

[0049] A series of 3×3 convolution modules are used to convolve the image to extract local features of the image. The convolution operation recognizes local patterns in the image through a series of filters.

[0050] The convolutional feature map is then downsampled through the maximum pooling layer (MaxPool) to reduce the spatial resolution of the feature map, reduce the size of the feature map, and maintain the feature information.

[0051] Feature fusion uses Transformer Blocks of different depths to extract feature vectors of different levels in the local feature map, and then concatenates the local feature map and feature vectors of different levels extracted by Transformer Block in the channel dimension to form a concatenated feature vector.

[0052] The Transformer Blocks of different depths include three Transformer Blocks, namely a first Transformer Block, a second Transformer Block and a third Transformer Block.

[0053] Each Transformer Block includes a multi-head attention mechanism, a local attention mechanism, and a Downsample operation.

[0054] The local attention mechanism in Transformer Block limits the attention calculation to a local window centered on the corresponding character. By limiting the scope of the attention mechanism, the model can pay more attention to the local strokes and structure of Chinese characters, which helps to distinguish Chinese characters with similar shapes.

[0055] Multi-head attention mechanism: allows the model to consider all elements when processing the entire sequence, which helps capture the global layout and arrangement rules of Chinese characters.

[0056] Downsample operation: concatenate adjacent vectors in the channel dimension to reduce the width of the feature map while maintaining the height to accommodate Chinese characters with different arrangements.

[0057] The Downsample operation in the Transformer Block concatenates two adjacent horizontally arranged vectors in the channel dimension.

[0058] The Downsample operation is disabled in the first TransformerBlock, and is enabled in the second and third TransformerBlocks.

[0059] The specific process of feature fusion is as follows:

[0060] The local feature map is extracted through the first Transformer Block to obtain the first feature vector;

[0061] The first feature vector is input into the second Transformer Block to extract features and obtain the second feature vector;

[0062] The second feature vector is input into the third Transformer Block to extract features and obtain the third feature vector;

[0063] The local feature map, the first feature vector and the second feature vector are linearly transformed through a fully connected layer and then concatenated into a fourth feature vector;

[0064] After the third eigenvector and the fourth eigenvector are concatenated, they are input into the Transformer Decoder for decoding.

[0065] The Chinese character recognition model architecture realizes the overall semantic feature extraction. The local feature map extracts the shallow feature representation, the first feature vector further extracts the feature, the second feature vector further extracts the feature of the first feature vector, and the third feature vector further extracts the deep feature representation. The feature vectors at different levels are spliced ​​to obtain the feature vector containing the overall semantics.

[0066] Decoding: Use Transformer Decoder to perform autoregressive decoding on the concatenated feature vector to generate the recognition result.

[0067] Specifically: The Decoder adopts a single-layer Transformer structure and takes the output of the encoder as input, using the Key and Value of the cross attention operation to generate the recognition result from left to right through autoregression.

[0068] When training the model of the present invention, a complete text sequence is sent to the Decoder, and masked attention is used to prevent "cheating".

[0069] When predicting, you can choose a greedy search or beam search strategy based on the probability distribution of the next character generated by the previous text.

[0070] Greedy search: selects the character with the highest probability as the next character, which is efficient but may fall into local optimality. Beam search: considers the k sequences with the highest scores, which is more comprehensive but consumes more computing resources. The score calculated by beam search is shown in formula (1):

[0071]

[0072] Where x is the encoder input, which can be the original language text in the NLP translation task or the input image. n-1 For the character at each position, p is the probability and S is the path score. As the path gets longer, the score gets lower. In order to avoid a more suitable long output text being eliminated by a short output text, a penalty based on the text length is usually added to S, as shown in formula (2), where length_penalty is a custom parameter.

[0073]

[0074] When using greedy search, the model stops generating when the end identifier is generated or the maximum generation length is reached. When using beam search, the model stops generating when the current highest-scoring sequence generates the end identifier or the maximum generation length is reached.

[0075] The evaluation indicators of this method use accuracy (ACC) and normalized edit distance (NED) to evaluate the model performance.

[0076] The input image is scaled to the same size, and then each pixel value in the image is divided by 255 to normalize it to the range of 0-1. The mean value is subtracted from the obtained value and divided by the standard deviation for standardization. The normalized image is passed through the CNN convolution layer and the maximum pooling layer MaxPool to extract the feature map.

[0077] In the described model, the convolutional layer (CNN) and the maximum pooling layer (MaxPool) together constitute the encoder part of the model, which is used to efficiently extract features from the input Chinese scene image. CNN uses local receptive fields and learnable convolution kernels to identify local patterns in the image, such as edges, textures, and shapes. These local features are crucial for distinguishing different Chinese character strokes. At the same time, the parameter sharing and translation invariance characteristics of CNN make the model insensitive to small position changes of the characters in the image, enhancing the generalization ability of the model. Then, the maximum pooling operation further abstracts the features by reducing the spatial resolution of the feature map. It retains the most significant visual features by selecting the maximum value in the local area while reducing the computational burden of the model. This downsampling process not only helps to simplify the model, but also increases the receptive field of the network, allowing the model to capture more global features, such as the structure and layout of the entire Chinese character. Through this feature extraction strategy combining CNN and MaxPool, this method can build a multi-level feature representation from low-level to high-level, providing rich visual information for subsequent TransformerBlock processing. These features are critical for accurately recognizing Chinese characters in images because they contain local details and global structures of Chinese characters. Overall, the collaborative work of CNN convolutional layers and MaxPool provides a solid feature extraction foundation for the entire text recognition model, enabling the model to efficiently process complex Chinese scene images and accurately recognize the text information therein.

[0078] Transformer Block plays a crucial role in the described model, responsible for in-depth processing and integration of image features. Each Transformer Block allows the model to focus on all elements in the sequence at the same time through the Self-Attention Mechanism, which significantly improves the model's ability to capture global image dependencies compared to traditional convolution operations that only rely on local information. In particular, the design of Multi-Head Attention enables the model to process information from multiple subspaces in parallel, enhancing the recognition and integration of different features. Local Attention focuses on capturing the local strokes and structure of Chinese characters. This design enables the model to distinguish Chinese characters with similar shapes and understand the contextual relationship of Chinese characters in the image when dealing with complex Chinese character recognition tasks. In addition, the Feed-Forward Neural Network (FFNN) in the Transformer Block further performs nonlinear transformations on the output of the self-attention, enhancing the model's expressive power. The application of residual connections and layer normalization ensures information flow and training stability in deep networks. Through these structures, Transformer Block can effectively extract and fuse image features, providing the model with rich information to generate accurate recognition results. In Chinese text recognition, this fine processing of local and global features is the key to improving recognition accuracy and robustness.

[0079] The combination of Transformer Block and CNN provides significant advantages for Chinese scene text recognition tasks in deep learning models. The CNN part uses its powerful local feature extraction capabilities to identify basic visual elements such as edges, textures, and shapes in images through convolutional layers and pooling layers. These local features are crucial for capturing the basic strokes and structure of Chinese characters. These features are then fed into the Transformer Block, which further processes these features through a self-attention mechanism, allowing the model to capture the relationship between strokes within Chinese characters and the interactions between Chinese characters.

[0080] The multi-head attention mechanism in Transformer Block allows the model to process information in different subspaces in parallel, which enhances the model's understanding of the diversity of image features and improves its ability to integrate local and global features of Chinese characters. In addition, the application of residual connections and layer normalization ensures the effective flow of information in deep networks, while stabilizing the training process and helping to avoid gradient vanishing or exploding problems.

[0081] Through this combination, the model can not only recognize local details in the image, but also understand how these details are combined into a meaningful whole in a larger context, which is especially important for Chinese character recognition because Chinese characters have a complex structure and there are a large number of characters with similar shapes. In addition, the advantage of this structure is also reflected in its ability to capture long-range dependencies, which is crucial for recognizing the order and contextual relationships of Chinese characters in images. Overall, the combination of CNN and Transformer Block provides a powerful feature extraction and sequence modeling framework for text recognition tasks in Chinese scenarios, significantly improving the accuracy and robustness of the model.

[0082] In the described method, the Transformer Decoder part plays the key role of transforming the image features extracted by the Encoder into the final text output. This process utilizes the self-attention mechanism, which allows the model to consider all previously generated characters in the sequence when generating each character, thereby capturing long-range dependencies, which is crucial for understanding and generating coherent text sequences. The autoregressive nature of the Decoder means that the generation of each character depends on the previous character, which mimics the natural process of humans when reading and writing. Structurally, the Decoder consists of multiple identical layers, each of which includes a self-attention layer, an encoder-decoder attention layer, and a feed-forward neural network. The self-attention layer enables the model to focus on the generated output, while the encoder-decoder attention layer allows the model to focus on the features provided by the Encoder. In addition, the application of residual connections and layer normalization ensures the effective flow of information in the deep network and helps to stabilize the training process. In terms of generation strategy, the Decoder can adopt greedy search or beam search. Greedy search is simple and efficient, generating sequences by selecting the character with the highest probability; while beam search considers multiple candidate characters and their combinations to find the optimal character sequence. This flexibility allows the model to balance computational efficiency and recognition accuracy according to actual needs.

[0083] In the described method, the loss function is a key part of the training process used to measure the difference between the model prediction and the actual annotation, which guides the optimization direction of the model parameters. The algorithm uses the CrossEntropy Loss function, which is a loss function commonly used in classification tasks, especially in natural language processing and text recognition.

[0084] The cross entropy loss function measures the difference between the probability distribution of the model output and the probability distribution of the true label. In the text recognition task, the output of the model is usually a probability distribution, where each position represents a possible character or symbol, and the true label is a one-hot encoded vector, where the correct character position is 1 and the rest are 0.

[0085] The calculation of the loss function can be expressed as:

[0086] L=-∑iyilog(pi) (3)

[0087] Where yi is the one-hot encoded vector of the true label, pi is the probability distribution of the model prediction, and L is the loss value. The cross entropy loss function quantifies the difference between the true label and the model prediction into a numerical value in the form of negative log-likelihood. During training, the goal is to minimize the value of the loss function by adjusting the model parameters.

[0088] In addition, since the longer the path, the lower the score will be, in order to avoid a more suitable long output text being eliminated by a short output text, a penalty term based on the text length is usually added to the loss function. This penalty term is a custom parameter used to balance the impact of text length on the loss function.

[0089] In the actual training process, the parameters of the model are updated through the back-propagation algorithm, using gradient descent or its variants (such as the Adam optimizer) to minimize the loss function. In this way, the model learns how to accurately map the features of the input image to the correct character sequence, thereby improving the accuracy of text recognition.

[0090] In the described algorithm, evaluation indicators are important tools to measure model performance. They provide quantitative standards for the quality of the model. The algorithm uses two main evaluation indicators: Accuracy (ACC) and Normalized Edit Distance (NED).

[0091] Accuracy (ACC): Accuracy is an intuitive evaluation indicator that measures the proportion of characters correctly recognized by the model to the total number of characters. In the text recognition task, accuracy directly reflects the recognition ability of the model.

[0092]

[0093] Where N is the total number of images in the test set, and δ is an indicator function that returns 1 when the prediction is equal to the true label, otherwise it returns 0. A high accuracy means that the model can correctly identify the characters in the image for most of the time.

[0094] Normalized edit distance (NED): Normalized edit distance is a more detailed evaluation metric that measures the difference between the model output and the true annotation. Edit distance refers to the minimum number of single-character edit (insertion, deletion, or substitution) operations required to transform one string into another. Normalized edit distance normalizes this value by dividing it by the maximum length of the input image characters so that the result can be between 0 and 1. The formula is:

[0095]

[0096] Where distancedistance represents the edit distance between the prediction result and the true label, and maxlenmaxlen is the maximum length of characters in all images. The lower the NED value, the smaller the difference between the model's prediction result and the true label, that is, the better the model's performance.

[0097] The type and source of the dataset used for training

[0098] Scene datasets: This type of dataset usually contains Chinese text images taken in various scenes, such as street scenes, billboards, signboards, etc. These images reflect the diversity and complexity of Chinese characters in the real world.

[0099] Web dataset: may contain Chinese text images captured from the Internet. These images may include web page screenshots, online documents, etc., reflecting the characteristics of text in the network environment.

[0100] Document datasets: This type of dataset may consist of scanned or photographed document pages that contain formatted text layouts, such as books, articles, reports, etc.

[0101] Dataset processing and use

[0102] Before using the dataset, the document mentioned that the images were resized to 32 pixels in height and 256 pixels in width, which helps the model process images from different sources in a consistent manner.

[0103] The performance of the model on different datasets is compared with the performance of other existing algorithms (such as CRNN, ASTER, MORAN, SAR, SEED, and TransOCR).

[0104] Fine-tuning your strategy

[0105] This method mentions a fine-tuning strategy, which is to first train the model on the Scene dataset and then fine-tune it on a dataset for a specific application scenario (such as store title recognition). This strategy helps the model better adapt to specific recognition tasks and improve recognition accuracy.

[0106] Ablation experiment

[0107] Ablation Study is an experimental method commonly used in machine learning and deep learning research to evaluate the contribution and importance of each component in the model. Through this method, researchers can understand which parts of the model have a significant impact on the final performance and which parts may be unnecessary. The model was subjected to ablation experiments on the Scene, Web, and Document datasets, removing the deep branch and local attention respectively. The figure lists three different experimental settings and gives their performance on different datasets:

[0108] w / o deep branch: indicates the model version without using deep learning branch. The performance on the Web, Document and Scene datasets is 64.12%, 67.31% and 95.77% respectively.

[0109] w / o local attention: represents the model version without using the local attention mechanism. The performance on the three datasets is 62.64%, 65.72%, and 95.53% respectively.

[0110] Ours: represents the complete model, including all features and techniques. The performance on the three datasets is 65.98%, 68.41% and 96.60% respectively.

[0111] The ablation experiment shows that the performance of "Ours" model on Document, Scene and Web datasets is improved compared with the model without deep branch and local attention.

[0112] From these data, as shown in Table 1, we can conclude that the deep learning branch and the local attention mechanism are beneficial to improving the performance of the model on specific datasets. The complete model (Ours) shows the best performance on all datasets, which shows that the synergy of various components in the model is effective in improving the overall recognition accuracy.

[0113] Table 1

[0114]

[0115] Comparative experiment

[0116] Comparative experiments are a method to quantify the performance of different models or algorithms by running and evaluating them under the same conditions. They can help researchers understand the advantages and limitations of each model and choose the most appropriate technical solution for a specific application scenario. In the field of machine learning and deep learning, comparative experiments usually involve multiple indicators such as accuracy, processing speed, and generalization ability to ensure a comprehensive evaluation of the performance of the model in practical applications.

[0117] As shown in Table 2, the comparative experiment involves CRNN, ASTER, MORAN, SAR, SEED, and TransOCR models, and experiments are conducted on Scene, Web, and Document datasets. The model achieves the best performance on the Web dataset, with a performance of 68.30 / 0.822, and also achieves good performance on the other two datasets, with performances of 65.99 / 0.802 and 96.60 / 0.995 respectively. The performance on the Scene dataset exceeds that of CRNN, ASTER, MORAN, SAT, and SEED models. The performance on the Document dataset exceeds that of MORAN, SAR, and SEED datasets. This comparative experiment proves that our model has performance advantages when dealing with text recognition tasks in different scenarios. Our model shows a high accuracy and a low edit distance, which means that our model has high accuracy and reliability in recognizing text on these datasets. However, it should be noted that the expression "exceeds" is relative to other models in the provided dataset, and does not mean the best performance among all possible models. In addition, other factors need to be considered in practical applications, such as model complexity, training time, and resource consumption.

[0118] Table 2

[0119]

[0120] This algorithm has shown significant advantages in the text recognition task in Chinese scenarios, mainly due to its adopted Transformer architecture, which is particularly suitable for processing sequence data and provides strong technical support for Chinese text recognition. By combining local and global attention mechanisms, the algorithm can accurately capture the local strokes and overall layout of Chinese characters, greatly improving the recognition accuracy of similar characters. In addition, the application of feature fusion technology enables the model to retain more Chinese character details, such as radicals or small strokes, further enhancing the recognition accuracy. The optimized design of the downsample operation ensures that while reducing the size of the feature map, key Chinese character information is not lost, especially for Chinese characters arranged vertically or diagonally, avoiding adverse effects. The provision of autoregressive output mode and greedy search and beam search generation strategies enables the model to strike a balance between accuracy and computational efficiency and adapt to different application requirements. The comprehensive application of evaluation indicators ACC and NED provides a quantitative evaluation of model performance. The introduction of fine-tuning strategies enables the model to quickly adapt to new application scenarios and enhances the practicality of the model. These advantages comprehensively reflect the efficiency, accuracy and adaptability of this algorithm in Chinese character recognition tasks, meet diverse recognition needs, and have high practical value and promotion potential.

[0121] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A Chinese character recognition method based on overall semantic feature extraction, characterized in that: Adopt Encoder-Decoder architecture, including: Get image: preprocess the input image; Feature extraction: The preprocessed input image is subjected to feature extraction and downsampling through a convolutional neural network to obtain local feature maps of the input image at different depth levels. The convolutional neural network includes a CNN convolutional layer and a MaxPool layer. Feature fusion: Use Transformer Blocks of different depths to extract feature vectors of different levels in the local feature map, and concatenate the local feature map and feature vectors of different levels extracted by Transformer Block in the channel dimension to form a concatenated feature vector; Decoding: Use Transformer Decoder to perform autoregressive decoding on the concatenated feature vector to generate the recognition result.

2. The Chinese character recognition method according to claim 1, characterized in that: The feature fusion step includes three Transformer Blocks, each of which includes a multi-head attention mechanism, a local attention mechanism and a Downsample operation.

3. The Chinese character recognition method according to claim 2, characterized in that: The three TransformerBlocks are respectively a first Transformer Block, a second Transformer Block and a third TransformerBlock; The local attention mechanism in Transformer Block limits the attention calculation to a local window centered on the corresponding character; The Downsample operation in the Transformer Block concatenates two adjacent horizontally arranged vectors in the channel dimension; The Downsample operation is disabled in the first TransformerBlock, and is enabled in the second and third TransformerBlocks.

4. The Chinese character recognition method according to claim 3, characterized in that: The feature fusion is specifically as follows: The local feature map is extracted through the first Transformer Block to obtain the first feature vector; The first feature vector is input into the second Transformer Block to extract features and obtain the second feature vector; The second feature vector is input into the third Transformer Block to extract features and obtain the third feature vector; The local feature map, the first feature vector and the second feature vector are linearly transformed through a fully connected layer and then concatenated into a fourth feature vector; After the third eigenvector and the fourth eigenvector are concatenated, they are input into the Transformer Decoder for decoding.

5. The Chinese character recognition method according to claim 1, characterized in that: The number of layers of Transformer Decoder is set to 1.

6. The Chinese character recognition method according to any one of claims 1 to 7, characterized in that: The performance of the text recognition method is evaluated using accuracy and normalized edit distance.

7. The Chinese character recognition method according to claim 1, characterized in that: The CNN convolutional layer includes multiple 3×3 convolutional modules.

8. The Chinese character recognition method according to claim 1, characterized in that: The local and first and second eigenvectors are converted into eigenvectors of uniform dimension through linear full connection, and the fourth eigenvector is not converted in dimension through the linear full connection layer.

9. The Chinese character recognition method according to claim 1, characterized in that: When training Transformer Decoder, the complete text sequence is fed into the Decoder, and masked attention is used to prevent "cheating"; When predicting, choose a greedy search or beam search strategy based on the probability distribution of the next character generated by the previous text.

10. The Chinese character recognition method according to claim 1, characterized in that: The recognition result is in text form.

Images