Zero-sample Chinese character recognition method based on radical structure tree graph level representation

Abstract

The invention discloses a zero-sample Chinese character recognition method based on radical structure tree graph level representation, and belongs to the technical field of Chinese character recognition. The method comprises the following steps: establishing a Chinese character database and constructing a directed radical structure tree of each Chinese character from top to bottom; adding direction position codes to nodes in the tree to distinguish the spatial relationship of brother nodes; inputting the radical structure tree with the direction and position codes into a GraphSAGE-based tree encoder to obtain normalized vector representation of the Chinese characters; extracting vector representation of the sample image by using ResNet; aligning the Chinese character vector and the image vector through comparative learning; and finally, zero sample recognition is realized by calculating the similarity between the to-be-recognized image and all candidate Chinese character vectors, and the highest similarity is selected as a recognition result. According to the method, the structure perception capability is improved through the directed tree, fine-grained spatial differences are perceived through directional coding, and the recognition capability of the model on unseen Chinese characters is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of Chinese character recognition technology, specifically to a zero-sample Chinese character recognition method based on a graph-level representation of a radical structure tree. Background Technology

[0002] Zero-shot Chinese character recognition is a key task in the field of optical character recognition. Its core definition is that a model can accurately recognize unseen Chinese characters during the prediction phase, even without encountering them during the training phase. The core objective of this task is to uncover the inherent structural and component information within Chinese characters, enabling the model to learn the compositional rules of known characters and thus possess the ability to generalize and recognize unseen characters.

[0003] Compared to traditional closed-set Chinese character recognition tasks, there are fundamental differences: traditional closed-set methods only need to identify the categories of Chinese characters already appearing in the training set, without needing to handle the recognition of new characters outside the training set; while zero-shot Chinese character recognition needs to overcome the limitation of "training data dependence," and still output accurate results when faced with unlabeled, unseen characters. This difference is particularly critical in Chinese character recognition scenarios. According to the national standard GB18030-2022, there are a total of 87,887 Chinese characters, but only about 3,755 commonly used first-level characters are used in daily communication. A large number of rare and low-frequency characters are difficult to collect sufficient labeled training data due to limited usage scenarios, causing traditional closed-set methods to be unable to cover the recognition needs of these types of characters. Therefore, zero-shot Chinese character recognition has become the core technical direction for solving the problem of "full coverage of Chinese character recognition."

[0004] However, zero-shot Chinese character recognition faces significant challenges: on the one hand, the internal structure of Chinese characters is complex, and there are a large number of Chinese characters with similar structures but completely different meanings, which can easily lead to model confusion; on the other hand, in the zero-shot recognition process, the model needs to determine the category of the Chinese character based on the image, and how to accurately extract the structural and component information of the Chinese character from the image is a key obstacle to achieving accurate recognition.

[0005] In summary, how to fully explore the structural information inside Chinese characters and construct a unified and accurate standardized representation based on the basic components of Chinese characters remains an important issue that urgently needs to be addressed in the field of zero-shot Chinese character recognition. Summary of the Invention

[0006] The purpose of this invention is to provide a zero-shot Chinese character recognition method based on the graph-level representation of the radical structure tree. By constructing a radical structure tree with direction awareness and encoding it into a normalized vector representation using a graph neural network, the model's ability to understand the hierarchical structure and fine-grained spatial relationships of Chinese characters is improved, ultimately achieving more accurate zero-shot Chinese character recognition.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A zero-shot Chinese character recognition method based on a graph-level representation of a radical structure tree includes the following steps:

[0009] Step S1: Establish a Chinese character database: The database contains Chinese characters and their corresponding semantic description sequences; the semantic description sequences are obtained by dividing the Chinese characters into a tree structure according to their structure and performing a preorder traversal.

[0010] Step S2: Construct a top-down radical structure tree: Based on the semantic description sequence, construct a corresponding tree structure for each Chinese character. This tree is a top-down radical structure tree, with the root node pointing to its child nodes, and each node having a self-loop.

[0011] Step S3: Add directional position codes to the radical structure tree: Assign a directional position code to each node in the tree, the directional position code including the root node, right node, middle node, and left node;

[0012] Step S4: Obtain the vector representation of Chinese characters: Input the radical structure tree with directional position encoding into the tree encoder for learning. The tree encoder includes a graph neural network layer and a global pooling layer. The tree encoder transforms the radical structure tree into a fixed-dimensional Chinese character vector.

[0013] Step S5: Obtain the vector representation of the sample image: Input the sample image into a convolutional neural network, extract image features, and convert them into an image vector with the same dimension as the Chinese character vector;

[0014] Step S6: Perform model training: Using a contrastive learning approach, calculate the similarity between the Chinese character vector and the image vector of the corresponding sample image, and use this similarity to train the model parameters;

[0015] Step S7: Perform Chinese character prediction task: Obtain the Chinese character vectors of all candidate Chinese characters and the image vector of the sample image to be predicted; calculate the similarity between the image vector of the sample image to be predicted and all candidate Chinese character vectors, and select the candidate Chinese character with the highest similarity as the recognition result.

[0016] Furthermore, in step S1, the root node and non-leaf nodes of the tree structure are structural components, and the leaf nodes are radical components.

[0017] Further, in step S3, the orientation position code is mapped to a vector through an embedding matrix, and the mapping formula is:

[0018] Position(v) = E pos [p v ],

[0019] in, For the embedding matrix, p v∈{1,2,3,4} represents the direction category index of node v. This represents the directional position encoding vector corresponding to node v.

[0020] Furthermore, in step S4, the graph neural network layer uses GraphSAGE, the tree encoder contains 5 layers of GraphSAGE, and the global pooling layer is a global average pooling layer, used to convert the output of the 5 layers of GraphSAGE into a 512-dimensional Chinese character vector.

[0021] Furthermore, in step S4, initial embedding is first performed on all nodes in the radical structure tree, and the calculation formula is as follows:

[0022]

[0023] Where v represents each node in the tree. The initial feature representation of node v is given by , Embedding(v) represents the vector representation of node v, and Position(v) represents the orientation position encoding vector of node v.

[0024] Furthermore, in step S4, only the feature update operation of GraphSAGE is used, and the sampling operation of its neighbor nodes is not used. In the k-th layer of GraphSAGE, the feature update formula for node v is:

[0025]

[0026] in, Denotes the set of neighbors of node v. The average representation of the features of the (k-1)th layer of neighboring nodes; concat(·) represents the average representation of the node's own features. splicing with features of its neighbors; W (k-1) σ is the weight matrix of the (k-1)th layer; σ(·) is the nonlinear activation function.

[0027] Further, step S5 specifically includes: first, using ResNet50 to extract image features from the sample images to obtain a feature map of dimension 8×8×2048, and then performing a global average pooling operation to obtain a vector of dimension 1×2048. The formula is:

[0028]

[0029] Among them, F resnet This represents the feature map output by ResNet50, and GAP(·) represents global average pooling.

[0030] Then, through two layers of feedforward neural network... The formulas for dimensionality reduction are as follows:

[0031]

[0032] in, Here is the weight matrix of the fully connected layer, b1 and b2 are bias terms, and σ(·) is the non-linear activation function. This is a 512-dimensional vector representation of the sample image.

[0033] Furthermore, in step S6, the contrastive learning is achieved by calculating cosine similarity, and its loss function is:

[0034]

[0035] in, and Let N represent the vector representation of the j-th Chinese character in a batch and the vector representation of the corresponding sample image, respectively, where N is the number of samples in the batch.

[0036] Further, in step S7, the image vector I and the candidate Chinese character vector T are calculated. i Cosine similarity s between i The formula is:

[0037]

[0038] Where i = 1, 2, ..., M, M is the number of candidate Chinese characters, and I·T i Let |·| denote the dot product of vectors, and |·| denote the L2 norm of the vector.

[0039] Furthermore, in step S7, the candidate Chinese character with the highest similarity is selected as the recognition result, using the following formula:

[0040]

[0041] Among them, i * It is the candidate Chinese character index with the highest similarity. This represents the candidate Chinese character that is most similar to the image vector I to be predicted.

[0042] The beneficial effects of the above scheme are as follows:

[0043] 1. This invention establishes a clear semantic transmission path, enhancing the ability to discriminate the semantic roles of radicals. By designing a top-down directed radical structure tree, the direction of information flow from the root node to its child nodes is clearly defined, and a self-loop is added to each node. This solves the problem of high-level semantics not being able to sink downwards, enabling the graph neural network to transmit semantics from high-level structures to low-level radicals. This makes the semantic role of radicals in the overall structure of Chinese characters clearer and effectively reduces the confusion rate of Chinese characters caused by the same radical but different structural levels. At the same time, the setting of node self-loops can preserve the original features of the node itself when the graph neural network aggregates neighbor features, avoiding the loss of core radical information due to only aggregating neighbor features, and ensuring the integrity of radical feature representation.

[0044] 2. This invention can explicitly distinguish differences in the position of radicals, enhancing the ability to recognize fine-grained structures of Chinese characters. This method introduces direction-aware positional encoding, mapping the positional information of four types of nodes—root, left, middle, and right—to fixed-dimensional vectors through an embedding matrix, and fusing it with the node's own features. This design allows the model to directly identify the positional relationships of sibling radicals under the same parent node, accurately capturing fine-grained structural differences in Chinese characters and reducing recognition errors caused by similar structures but different radical positions. For low-quality Chinese character images with irregular writing and blurred strokes, direction-aware positional encoding can provide explicit semantic labels for radical positions, assisting the model in judging the relative relationships of radicals and improving the model's robustness in recognizing low-quality images.

[0045] 3. This invention effectively improves zero-shot generalization ability. Through a fusion design of 5-layer GraphSAGE, global pooling, and contrastive learning, it achieves direct alignment of Chinese character structural semantics and visual features within the same vector space. Compared to methods such as semantic vector mean square error optimization, indirect alignment using generated samples, or sequence filtering, this alignment method more fully utilizes the hierarchical information of Chinese character structure, strengthens the correlation between structural semantics and visual features, and significantly improves the model's recognition accuracy for unseen Chinese characters, especially demonstrating superior generalization ability in zero-shot scenarios.

[0046] 4. This invention reduces reliance on training data and adapts to the needs of rare character recognition. This method learns the rules and structural features of Chinese character radical combinations, replacing the reliance on massive amounts of labeled data found in traditional methods. Faced with the problem of a vast number of Chinese characters and the difficulty in collecting sufficient training data for a large number of rare characters, the model does not need to acquire training data separately for each rare character. It only needs to generate corresponding vectors based on the radical composition and structure tree of the rare character, and then match them with image features to complete the recognition. Furthermore, for Chinese character images with diverse styles and a high proportion of rare characters, the model can leverage the learned common radical embedding features, combined with the directional encoding of the structure tree, to generalize to the recognition of new radicals, effectively covering the rare character recognition needs that traditional methods struggle to address. Attached Figure Description

[0047] Figure 1This is a flowchart illustrating the overall structure of the present invention;

[0048] Figure 2 This is a schematic diagram of directional position encoding in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the radical structure tree from top to bottom in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the radical structure tree from bottom to top in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of a bidirectional radical structure tree in an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of 12 structural representations of Chinese characters in an embodiment of the present invention. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0054] It should be noted that, unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0055] This embodiment provides a zero-shot Chinese character recognition method based on a graph-based radical structure tree representation (GRSTR). It constructs a radical structure tree with direction awareness and transforms it into a normalized vector representation using a graph neural network. Finally, it achieves accurate matching between the image and the Chinese character structure through contrastive learning. Figures 1 to 6 As shown, the method includes the following steps:

[0056] Step S1: Establish a Chinese character database

[0057] This database contains all the Chinese characters to be identified and their corresponding semantic description sequences. Each character is divided according to its 12 inherent basic structural types, forming a unique tree structure representation. By performing a preorder traversal of this tree structure, the semantic description sequence of the Chinese character can be obtained. In this tree structure, the root node and all non-leaf nodes represent the structural components of the Chinese character, while the leaf nodes represent the specific radical components. The specific operation is as follows:

[0058] Reference Appendix Figure 6, according to the 12 basic structures of Chinese characters, including left-right, up-down, enclosed, single-component, etc. structures, each Chinese character is disassembled into a tree structure. Structural components such as the nodes corresponding to "left-right structure" and "up-down structure", and "radical components" such as basic radicals like "wood", "water radical", "person", etc.

[0059] Adopt the pre-order traversal rule to traverse the above tree structure: starting from the root node, first visit the current node, and then sequentially traverse the child nodes of the current node. After the traversal is completed, a unique semantic description sequence of the Chinese character is generated. For example, the tree structure of the Chinese character is: left-right structure - left radical "water radical" - right radical "wood". After pre-order traversal, the semantic description sequence obtained is "left-right structure, water radical, wood".

[0060] Correspondingly store the "Chinese character character - tree structure - semantic description sequence" of all Chinese characters to form a complete Chinese character database, which serves as the basic data for constructing the radical structure tree subsequently.

[0061] Step S2: Construct a top-down radical structure tree [[ID=1l]]

[0062] According to the semantic description sequence obtained in step S1, construct a top-down directed radical structure tree for each Chinese character, refer to the appendix Figure 3 . The edges of this tree have clear directions, pointing from the root node to its child nodes, which defines the flow direction of structural information in the graph neural network. In addition, in order to ensure that the information of each node can be effectively transmitted and updated in the graph neural network, a self-loop is added to each node in the tree.

[0063] As Figure 2 shown, taking the Chinese character "竣" as an example, its semantic description sequence can be disassembled into a sequence containing high-level structures and low-level radicals. In the constructed top-down radical structure tree, the topmost node corresponds to the overall structure type of the Chinese character "竣". This node points to the two child nodes "立" and "发" through directed edges. The "发" node points to the three lower-level child nodes "厶", "八", and "夂" through directed edges. Moreover, the nodes of the overall structure type of the Chinese character "竣", the "立" node, the "发" node, the "厶" node, the "八" node, and the "夂" node all have self-loops respectively, completing the construction of the radical structure tree of the Chinese character "竣", clearly presenting the "top-down" structural information transmission path.

[0064] Step S3: Add direction position encoding to the radical structure tree

[0065] In order to distinguish the spatial relative position relationships of different child nodes under the same parent node, the present invention introduces direction position encoding for each node in the tree. This encoding has four types, corresponding to the root node, right node, middle node, and left node respectively. The direction position encoding refers to Figure 2 .

[0066] In practice, a learnable embedding matrix maps the orientation category index to a 512-dimensional vector. For a node v, its orientation position encoding vector is calculated using the mapping formula:

[0067] Position(v) = E pos [p v ],

[0068] in, For the embedding matrix, p v ∈{1,2,3,4} represents the direction category index of node v. This represents the directional position encoding vector corresponding to node v.

[0069] The directional position encoding vector is then added bitwise to the embedding vector of node v itself to obtain the tree encoder of the initialization feature input step S4 for node v.

[0070] Step S4: Obtain the vector representation of Chinese characters.

[0071] The radical structure tree with directional position encoding is input into the tree encoder to obtain a normalized vector representation of each Chinese character. The tree encoder consists of 5 GraphSAGE layers and 1 global average pooling layer. GraphSAGE only retains feature update operations and does not use neighbor sampling operations. The reason for not sampling neighbors is that the number of nodes in the Chinese character radical structure tree is small, and full neighbor aggregation can ensure the integrity of structural information.

[0072] The specific implementation process is as follows:

[0073] (1) Perform initial embedding on all nodes in the tree, calculated using the following formula:

[0074]

[0075] Where v represents each node in the tree. The initial feature representation of node v is given by , Embedding(v) represents the vector representation of node v, and Position(v) represents the orientation position encoding vector of node v.

[0076] (2) Input the initialized tree structure into a 5-layer GraphSAGE for feature learning. Each layer updates node features using the following formula:

[0077]

[0078] in, Denotes the set of neighbors of node v. The average representation of the features of the (k-1)th layer of neighboring nodes; concat(·) represents the average representation of the node's own features. splicing with features of its neighbors; W (k-1) σ is the weight matrix of the (k-1)th layer; σ(·) is the ReLU nonlinear activation function.

[0079] In each layer, each node updates its own features by aggregating the features of its neighboring nodes. The mean aggregation method can effectively integrate local structural information.

[0080] (3) Input all node features output by the 5-layer GraphSAGE into the global average pooling layer, calculate the average value of all node features, and obtain the 512-dimensional vector of the Chinese character. This vector is the normalized representation of the Chinese character, which completes the conversion from tree to vector.

[0081] Step S5: Obtain the vector representation of the sample image.

[0082] The sample image is input into a convolutional neural network to extract image features and convert them into an image vector with the same dimensions as the Chinese character vector. The specific implementation is as follows:

[0083] (1) ResNet50 feature extraction: ResNet50 is used to extract image features from the sample images, resulting in a feature map F with dimensions of 8×8×2048. resnet This feature map contains high-level visual information about the image.

[0084] (2) Global average pooling: Perform global average pooling on the feature map to transform the 8×8×2048 feature map into a 1×2048 vector. The formula is:

[0085]

[0086] Among them, F resnet This represents the feature map output by ResNet50, and GAP(·) represents global average pooling.

[0087] (3) Two-layer feedforward neural network dimensionality reduction: To make the dimensions of the image vector consistent with those of the Chinese character vector, a two-layer fully connected feedforward neural network is constructed. The formulas for dimensionality reduction are as follows:

[0088]

[0089] in, Here is the weight matrix of the fully connected layer, b1 and b2 are bias terms, and σ(·) is the non-linear activation function. This is a 512-dimensional vector representation of the sample image.

[0090] Step S6: Perform model training

[0091] A contrastive learning approach is used to calculate the similarity between the Chinese character vector and the corresponding sample image vector, and this similarity is used to train the model parameters; this aligns the two types of vectors in the same feature space, as detailed below:

[0092] For N samples in a training batch, the image vector I is calculated using cosine similarity. j With candidate Chinese character vector T i Maximize similarity. The loss function is:

[0093]

[0094] in, and Let N represent the vector representation of the j-th Chinese character in a batch and the vector representation of the corresponding sample image, respectively, where N is the number of samples in the batch.

[0095] The parameters of the tree encoder ResNet50 and the feedforward neural network are integrated into a unified model to... To optimize the objective, the Adam optimizer is used for iterative training until the loss function converges.

[0096] Step S7: Perform the Chinese character prediction task

[0097] Obtain the character vectors of all candidate Chinese characters and the image vector of the sample image to be predicted; calculate the similarity between the image vector of the sample image to be predicted and the vectors of all candidate Chinese characters, and select the candidate Chinese character with the highest similarity as the recognition result. Specifically, this includes:

[0098] (1) Preprocessing of candidate Chinese character vectors: Collect all candidate Chinese characters, and generate a 512-dimensional vector for each candidate Chinese character according to the process of steps S2-S4, forming a "candidate Chinese character-vector" mapping table, denoted as {T i ∣i=1,2,…,M}, where M is the number of candidate Chinese characters.

[0099] (2) Extract the image vector to be predicted: Input the Chinese character image to be predicted into the trained ResNet50 and feedforward neural network according to the process of step S5 to obtain a 512-dimensional image vector I.

[0100] (3) Calculate the cosine similarity: Calculate the similarity between I and the vector T of each candidate Chinese character. i The cosine similarity is calculated using the following formula:

[0101]

[0102] Among them, I·T iLet |·| denote the dot product of vectors, and |·| denote the L2 norm of the vector.

[0103] (4) Determine the recognition result: Select the candidate Chinese character with the highest similarity as the final recognition result, using the following formula:

[0104]

[0105] Among them, i * It is the candidate Chinese character index with the highest similarity, T i * indicates the candidate Chinese character that is most similar to the image vector I to be predicted.

[0106] Implementation effect verification

[0107] To verify the effectiveness of this method, comparative and ablation experiments were conducted on publicly available datasets:

[0108] The experiment was conducted using the PyTorch deep learning framework and implemented in Python. The training environment consisted of a single NVIDIA A30 Tensor Core GPU with 24GB of VRAM, and the initial learning rate was 1×10⁻⁶. -4 The Adam optimizer was used, with momentum parameters β1 and β2 set to 0.9 and 0.98 respectively, and a batch size of 128. To ensure fair comparison with existing technologies, the resolution of all original Chinese character images was uniformly adjusted to 32×32 pixels before training. The datasets used were HWDB 1.0-1.1, ICDAR2013, and the AncientAHCDB public dataset. The graph neural network in the tree encoder was set to 5 layers with a hidden layer dimension of 512. Specific dataset descriptions and experiments are as follows:

[0109] (1) Dataset:

[0110] The HWDB 1.0-1.1 dataset contains 2,678,424 handwritten Chinese character images provided by 720 writers, covering 3,881 character categories; this dataset includes 3,755 commonly used first-level characters from the GB 2312 standard. Due to the large number of writers and significant differences in handwriting styles, character recognition faces considerable challenges. The ICDAR2013 dataset contains 224,419 handwritten Chinese character images, covering 3,755 character categories, consistent with the commonly used first-level characters. The AncientAHCDB dataset, collected from ancient Chinese texts, contains a large number of rare and obsolete characters, significantly increasing the difficulty of recognition. This dataset includes two subsets: AHCDB-style1 contains clear character images with neat handwriting and a relatively uniform style, while AHCDB-style2 contains low-quality images with diverse and irregular handwriting styles. The base categories from both subsets are used as training data, and the test set corresponding to the augmented categories is selected as evaluation data. Specifically, the AHCDB-style1 training set contains 832,939 images, and the test set contains 36,258 images; the AHCDB-style2 training set contains 728,423 images, and the test set contains 19,597 images. Both training sets share the same 2,365 character categories, while the AHCDB-style1 test set covers 3,277 categories, and the AHCDB-style2 test set contains 783 categories.

[0111] (2) Evaluation Metric: Accuracy is used as the evaluation metric, defined as follows:

[0112]

[0113] Where, N true N represents the samples that were correctly predicted. false N represents the samples that were predicted incorrectly. total This represents all samples.

[0114] (3) Comparative experiment:

[0115] The training set is composed of the first M characters selected from the 3755 characters in the HWDB1.0-1.1 dataset, where M can take values ​​of {500, 1000, 1500, 2000, 2755}. The test set is selected from the ICDAR2013 dataset and consists of the last 1000 characters.

[0116] Table 1 shows a comparative experiment using the HWDB 1.0-1.1 dataset and the ICDAR 2013 dataset.

[0117] method M＝500 M＝1000 M＝1500 M＝2000 M＝2755 average DenseRAN 1.70％ 8.44％ 14.71％ 19.51％ 30.68％ 15.01％ HDE 4.90％ 12.77％ 19.25％ 25.13％ 33.49％ 19.11％ STAR 7.54％ 19.47％ 27.79％ 35.53％ 43.86％ 26.84％ CUE 7.43％ 15.75％ 24.04％ 27.04％ 40.55％ 22.96％ CCR-CLIP 21.79％ 42.99％ 55.86％ 62.99％ 72.98％ 51.32％ FT-CLIP 26.17％ 50.68％ 62.25％ 70.39％ 78.96％ 57.69％ GRSTR (This invention) 32.87％ 56.06％ 67.74％ 73.41％ 80.62％ 62.14％

[0118] As shown in Table 1, the GRSTR model proposed in this invention exhibits excellent performance on the handwritten Chinese character dataset under zero-sample settings. Compared with existing methods, the GRSTR model has significant performance advantages: compared with the CCR-CLIP method, the average accuracy of the GRSTR model is improved by 10.82%; compared with the FT-CLIP method, the average accuracy of the GRSTR model is improved by 4.45%.

[0119] Specifically, let's examine the performance differences under different training set sizes: when the training set size M is 500, 1000, 1500, 2000, and 2755, the accuracy improvement of the GRSTR model compared to the FT-CLIP method is 6.70%, 5.38%, 5.49%, 3.02%, and 1.66%, respectively. These results indicate that the performance of all methods improves with increasing training set size, but the performance advantage of the GRSTR model over other methods gradually decreases as the training set size increases.

[0120] In scenarios with limited training data, the GRSTR model achieves a more significant performance improvement. This advantage mainly stems from the powerful generalization ability of graph neural networks. By accurately capturing the structural information of Chinese characters and the rules of radical combination, the GRSTR model can efficiently transfer learning experience when data is scarce. As the amount of training data increases, the number of radical categories in the input tree encoder expands accordingly, which to some extent offsets the performance degradation caused by insufficient radical samples when training data is scarce. At the same time, the greater diversity and quantity of radicals in large-scale training datasets allows the performance of other data-driven methods to gradually approach that of the GRSTR model, ultimately narrowing the performance gap between the GRSTR model and other methods.

[0121] Table 2 shows the comparative experiments on the AHCDB dataset.

[0122] method backbone network AHCDB-Style1 AHCDB-Style2 DenseRAN DenseNet-a 36.1％ 14.7％ HDE ResNet-a 39.3％ 20.7％ SideNet-DDCM-a ResNet-a 46.2％ 29.2％ SideNet-DDCM-c ResNet-c 49.6％ 36.0％ SideNet-DDCM DenseNet-b 52.4％ 38.2％ GRSTR-c ResNet-c 70.36％ 43.31％ GRSTR (This invention) ResNet-b 73.05％ 45.84％

[0123] As shown in Table 2, the model of this invention exhibits significant performance advantages on both sub-datasets of the AHCDB dataset (AHCDB-Style1 and AHCDB-Style2). Specifically, the GRSTR-c model based on ResNet-c achieves a 20.76% improvement in accuracy on the AHCDB-Style1 dataset compared to the SideNet-DDCM-c model, and a 7.31% improvement in accuracy on the AHCDB-Style2 dataset compared to the SideNet-DDCM-c model. The core GRSTR model proposed in this invention achieves an accuracy of 73.05% on the AHCDB-Style1 dataset and 45.84% on the AHCDB-Style2 dataset, representing improvements of 20.65% and 7.64% respectively compared to the SideNet-DDCM model using the DenseNet-b encoder.

[0124] The comparative experimental results above fully demonstrate that the GRSTR method proposed in this invention is highly competitive in Chinese character recognition tasks, and its performance is significantly better than the existing SideNet-DDCM series methods. Therefore, this invention effectively improves the accuracy of Chinese character recognition by accurately mining the structural information of Chinese characters, explicitly distinguishing the semantics of radical positions, and efficiently aligning structural semantics with image features. It performs particularly well in Chinese character image recognition scenarios of different qualities and styles, effectively solving the performance bottlenecks of existing technologies and possessing clear and significant beneficial effects.

[0125] (3) Ablation experiment:

[0126] To verify the "top-to-bottom radical structure tree" of this invention (e.g.) Figure 3 The effectiveness of (as shown) can be generalized to two structural variants: such as Figure 4 The "bottom-to-top radical structure tree" shown is similar to... Figure 5 The "bidirectional radical structure tree" is shown, and the impact of the "top-down radical structure tree" and "direction position encoding" on the model performance is analyzed through comparative experiments on the HWDB1.0-1.1 dataset and the ICDAR2013 dataset. The experimental results are shown in Table 3.

[0127] Where A represents the "top-down radical structure tree", B represents the "bottom-up radical structure tree", C represents the "bidirectional radical structure tree", and D represents the "directional position code".

[0128] Table 3 Ablation Experiments of "Top-Down Radical Structure Tree" and "Directional Position Code"

[0129]

[0130] As shown in Table 3, when using the "top-down radical structure tree," the average accuracy without "directional position encoding" is 55.32%, while the average accuracy with "directional position encoding" is 62.14%, demonstrating the effectiveness of the "directional position encoding" proposed in this invention. When using "directional position encoding," the average accuracy of the "top-down radical structure tree" is 62.14%. However, the average accuracies of the "bottom-up radical structure tree" and the "bidirectional radical structure tree" are 59.68% and 60.75%, respectively, further demonstrating the effectiveness of the "top-down radical structure tree" proposed in this invention.

[0131] Table 4 shows the ablation experiments of neural network variants, dimensions, and layers.

[0132] Dimension number of floors GraphSAGE GCN GAT GIN 256 4 80.56％ 79.77％ 79.61％ 79.04％ 512 5 80.62％ 79.54％ 79.08％ 77.83％

[0133] To verify the impact of graph neural network parameters on model performance, this invention conducts ablation experiments on the HWDB1.0-1.1 dataset, focusing on graph neural network variants, feature dimensions, and the number of network layers. The experimental results are shown in Table 4.

[0134] As shown in Table 4, GraphSAGE consistently exhibits optimal performance under different parameter combinations. When the number of graph neural network layers is set to 4 and the feature dimension to 256, GraphSAGE demonstrates the best performance. Even when adjusting the parameter combination to increase the number of layers to 5 and the feature dimension to 512, GraphSAGE still maintains optimal performance, further proving that in the zero-shot Chinese character recognition scheme of this invention, GraphSAGE is the most adaptable graph neural network variant, and it can stably demonstrate its advantages under different layer and dimension settings.

[0135] In summary, this embodiment effectively solves the problems of unclear structural information transmission, difficulty in distinguishing positional differences, and inaccurate feature alignment in zero-sample Chinese character recognition through the core design of "top-down radical structure tree", "direction-aware positional encoding", and "fusion of GraphSAGE and contrastive learning", thus achieving efficient recognition of unseen Chinese characters.

[0136] Finally, it should be noted that any parts of this invention not described in detail are prior art. Those skilled in the art will understand that the above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A zero-shot Chinese character recognition method based on a radical structure tree graph-level representation, characterized in that, Includes the following steps: Step S1: Establish a Chinese character database: The database contains Chinese characters and their corresponding semantic description sequences; the semantic description sequences are obtained by dividing the Chinese characters into a tree structure according to their structure and performing a preorder traversal. Step S2: Construct a top-down radical structure tree: Based on the semantic description sequence, construct a corresponding tree structure for each Chinese character. This tree is a top-down radical structure tree, with the root node pointing to its child nodes, and each node having a self-loop. Step S3: Add directional position codes to the radical structure tree: Assign a directional position code to each node in the tree, the directional position code including the root node, right node, middle node, and left node; Step S4: Obtain the vector representation of Chinese characters: Input the radical structure tree with directional position encoding into the tree encoder for learning. The tree encoder includes a graph neural network layer and a global pooling layer. The tree encoder transforms the radical structure tree into a fixed-dimensional Chinese character vector. Step S5: Obtain the vector representation of the sample image: Input the sample image into a convolutional neural network, extract image features, and convert them into an image vector with the same dimension as the Chinese character vector; Step S6: Perform model training: Using a contrastive learning approach, calculate the similarity between the Chinese character vector and the image vector of the corresponding sample image, and use this similarity to train the model parameters; Step S7: Perform Chinese character prediction task: Obtain the Chinese character vectors of all candidate Chinese characters and the image vector of the sample image to be predicted; calculate the similarity between the image vector of the sample image to be predicted and all candidate Chinese character vectors, and select the candidate Chinese character with the highest similarity as the recognition result.

2. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 1, characterized in that, In step S1, the root node and non-leaf nodes of the tree structure are structural components, and the leaf nodes are radical components.

3. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 1, characterized in that, In step S3, the orientation position code is mapped to a vector through an embedding matrix, and the mapping formula is: Position(v)=E pos [p v ], in, For the embedding matrix, p v ∈{1,2,3,4} represents the direction category index of node v. This represents the directional position encoding vector corresponding to node v.

4. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 1, characterized in that, In step S4, the graph neural network layer uses GraphSAGE, the tree encoder contains 5 layers of GraphSAGE, and the global pooling layer is a global average pooling layer, which is used to convert the output of the 5 layers of GraphSAGE into a 512-dimensional Chinese character vector.

5. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 4, characterized in that, In step S4, initial embedding is first performed on all nodes in the radical structure tree, and the calculation formula is as follows: Where v represents each node in the tree. The initial feature representation of node v is given by , Embedding(v) represents the vector representation of node v, and Position(v) represents the orientation position encoding vector of node v.

6. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 5, characterized in that, In step S4, only the feature update operation of GraphSAGE is used, and the sampling operation of its neighbor nodes is not used. In the k-th layer of GraphSAGE, the feature update formula of node v is: in, Denotes the set of neighbors of node v. The average representation of the features of the (k-1)th layer of neighboring nodes; concat(·) represents the average representation of the node's own features. splicing with features of its neighbors; W (k-1) σ is the weight matrix of the (k-1)th layer; σ(·) is the nonlinear activation function.

7. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 1, characterized in that, Step S5 specifically includes: First, using ResNet50 to extract image features from the sample images to obtain a feature map of dimension 8×8×2048, and then performing global average pooling to obtain a vector of dimension 1×2048. The formula is: Among them, F resnet This represents the feature map output by ResNet50, and GAP(·) represents global average pooling. Then, through two layers of feedforward neural network... The formulas for dimensionality reduction are as follows: in, Here is the weight matrix of the fully connected layer, b1 and b2 are bias terms, and σ(·) is the non-linear activation function. This is a 512-dimensional vector representation of the sample image.

8. The zero-sample Chinese character recognition method based on radical structure tree graph representation according to claim 1, characterized in that, In step S6, the contrastive learning is achieved by calculating cosine similarity, and its loss function is: in, and Let N represent the vector representation of the j-th Chinese character in a batch and the vector representation of the corresponding sample image, respectively, where N is the number of samples in the batch.

9. The zero-sample Chinese character recognition method based on a radical structure tree graph-level representation according to claim 1, characterized in that, In step S7, the image vector I and the candidate Chinese character vector T are calculated. i Cosine similarity s between i The formula is: Where i = 1, 2, ..., M, M is the number of candidate Chinese characters, and I·T i Let |·| denote the dot product of vectors, and |·| denote the L2 norm of the vector.

10. The zero-sample Chinese character recognition method based on a radical structure tree graph-level representation according to claim 1, characterized in that, In step S7, the candidate Chinese character with the highest similarity is selected as the recognition result, using the following formula: Among them, i * It is the candidate Chinese character index with the highest similarity. This represents the candidate Chinese character that is most similar to the image vector I to be predicted.

Images