Defective character recognition method based on relational distillation graph network

Abstract

The invention provides a damaged character recognition method based on a relational distillation graph network, and the method comprises the steps: introducing a radical self-information module based on a teacher model, carrying out the GNN graph neural network optimization of a node feature vector, introducing a radical self-information module into a student model, carrying out the MLP multi-layer perceptron network optimization of the node feature vector, and carrying out the recognition of a damaged character. Constructing a relational distillation graph network model, carrying out distillation training on the student model by adopting direct distillation and relational distillation, and transmitting output probability distribution of the teacher model to the student model through a distillation process, so that the student model learns relational reasoning and structural knowledge of the teacher model; the method aims at solving the challenges of noise transmission, model recognition efficiency, cross-model knowledge distillation and the like in the recognition and understanding process of damaged Chinese characters in ancient inscription inscriptions in a graph neural network, so that higher accuracy and robustness are realized in the repair and recognition process of damaged characters.

Description

Technical Field

[0001] The present invention relates to the technical field of damaged text recognition, and in particular to a damaged text recognition method based on a relational distillation graph network. Background Art

[0002] Currently, research on damaged text recognition consists of two main steps: first, text restoration, and second, recognition. Mainstream text restoration methods typically use image inpainting algorithms to restore the integrity of text, and then subject the restored text to traditional recognition methods. Many applications, such as ancient inscriptions and books, lack data understanding and cannot utilize context for damaged text restoration and recognition. Therefore, DynamicGrape utilizes damaged text components for recognition. This method decomposes each Chinese character into a tree structure and encodes it into a semantic vector. It then constructs a special graph structure based on a bipartite graph to effectively model the relationship between Chinese characters, structures, and components. By incorporating edge embedding into the GraphSAGE graph neural network, the node information aggregation and transfer mechanism is used to deeply explore the internal features of Chinese characters, optimize the node representation of damaged characters, and supplement the encoding of potentially missing components or structures with an edge prediction module, ultimately achieving accurate inference of damaged characters. By incorporating prior knowledge of text structure, this method transforms the damaged text recognition task into the recognition of a component set. This strategy effectively alleviates the recognition pressure of a large number of character categories while expanding the number of training samples through component reorganization. This method provides a new technical path for damaged text recognition research, but many problems still exist: (1) The model has weak generalization capabilities. Because the DynamicGrape model input includes components in addition to text images, the model has insufficient generalization capabilities in unknown character or cross-domain font recognition tasks because no clear component information is provided. Moreover, when the degree of damage is high, the input component category set lacks sufficient prompt information, and the graph network cannot accurately aggregate neighbor features to optimize the representation of the text nodes to be recognized, resulting in poor performance in downstream recognition tasks.

[0003] (2) Dynamically adding new nodes introduces noise. The GNN model is a neighbor-dependent network structure, and node features are heavily dependent on neighbor aggregation. When new damaged Chinese nodes are added to the graph, since damaged Chinese nodes often contain missing or inaccurate edges, the process of aggregating neighboring node information through the message passing mechanism of the graph network will cause the neighbor features to be contaminated, thereby destroying the stability of the original graph structure.

[0004] (3) The graph structure is highly complex and inefficient. As the graph scales, the number of connected edges increases exponentially, resulting in high inference costs and poor real-time performance. The current dataset contains 2,527 first-level Chinese characters, and the bipartite graph constructed from them already has tens of thousands of connected edges. If it is expanded to second-level, third-level, and even traditional Chinese characters, the size of the graph will increase significantly, leading to a significant increase in the computational cost of model training and inference, and a decrease in the aggregation and update efficiency of the graph network, thus affecting real-time recognition capabilities. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the problems existing in the above-mentioned prior art and provide a damaged text recognition method based on a relational distillation graph network, which aims to solve the challenges of noise transmission, model recognition efficiency and cross-model knowledge distillation in the process of graph neural network recognition and understanding of damaged Chinese characters in ancient books and inscriptions, thereby achieving higher accuracy and robustness in the process of repairing and recognizing damaged text.

[0006] To this end, the present invention adopts the following technical solution: a damaged text recognition method based on a relational distillation graph network, which includes: introducing a radical self-information module based on a teacher model, and performing GNN graph neural network optimization on the node feature vector, so that the teacher model has a bipartite graph structure G and a node feature vector X-SIR with radical self-information; introducing a radical self-information module based on a student model, and performing MLP multi-layer perceptron network optimization on the node feature vector, so as to obtain the node feature vector X-SIR of the student model with radical self-information; constructing a relational distillation graph network model, and using direct distillation and relational distillation to perform distillation training on the student model, and passing the output probability distribution of the teacher model to the student model through the distillation process, so that the student model learns the relational reasoning and structural knowledge of the teacher model; the total loss function in the above distillation process includes the loss function in the teacher model and the loss function for constraining the student during the distillation process.

[0007] Furthermore, the step of introducing the radical self-information module based on the teacher model includes: Input, first count the components or structures The probability of appearing in 2527 first-level Chinese characters, and then calculating the radical self-information, is shown in the following formula: ,in, For components or structures The amount of self-information, For components or structures The probability of appearing in 2527 characters; after obtaining the radical self-information, the "1" in the one-hot initial feature vector of the original component node in the bipartite graph structure is replaced with the corresponding self-information value, and the teacher model is obtained as having a bipartite graph structure G and a node feature vector X-SIR with radical self-information.

[0008] Furthermore, the direct distillation step comprises: first The representation output of the second layer and student models Node representation output generated by multi-layer perceptron network optimization The mean squared error loss is calculated to measure the similarity between the student model and the teacher model in terms of node features, as shown in the following formula: ,in, Represents all nodes in the graph structure, Indicates the use of mean square error loss, i Indicates a component or word structure; The node feature vector input by the teacher model is then optimized by the GNN graph neural network. The edge prediction module is used to obtain the new edge relationship D ̂uv between the text node to be identified and the component and structure node on the optimized vector node representation, thereby constructing the corresponding encoding (hde) ̅_char of the text to be identified and obtaining the loss function of the logical matching. : ,in, Indicates the set that needs to be filled in the edge prediction module; represents the edge weight predicted by the edge prediction module in the teacher model; Represents the edge weight predicted by the edge prediction module in the student model; To use mean square error loss, i, j Indicates a component or word structure; Finally, the outputs of the teacher model and the student model in the edge prediction module are aligned, so that the student model can learn the relational reasoning and structural knowledge of the teacher model.

[0009] Furthermore, the relationship distillation step includes: introducing an independent "radical relevance score" , the edge weights output by the edge prediction module are subjected to probability normalization and converted into standardized values ​​representing the predicted probability of connections between nodes. Specifically: the edge weights of the teacher model are obtained by Function normalization yields , as a probabilistic representation of its radical association; the student model uses the same method to generate the corresponding score , both are jointly optimized through sorting and distribution consistency in the distillation stage.

[0010] Furthermore, the step of relation distillation further includes: calculating the predicted probability of the student model output The difference between the two context nodes predicted by the student model is combined with the information of the teacher model to determine the central node 𝑣. and Who is responsible for the node The relative ordering relationship between point pairs enables the student model to learn the relationship knowledge between nodes.

[0011] Furthermore, the loss function in the teacher model is As shown in the following formula: ,in is the binary classification loss of the damage judgment module, is the mapping loss of the bidirectional embedding transfer module, is the reconstruction loss of the bidirectional embedding transfer module, is the prediction loss of the weighted edge prediction module, , , , are the weight hyperparameters of each loss term.

[0012] Furthermore, the loss function constraining the student during the distillation process is It is composed of the losses of four matching strategies as follows: , where To represent the matching loss, is the logistic matching loss, is the ranking matching loss, is the distribution matching loss, , , , Different weights are assigned to each loss.

[0013] The beneficial effects of the present invention are as follows: a radical self-information module is introduced into the teacher model and the student model by encoding the information of rare radicals into the initial feature vector; secondly, in order to solve the problem of decreased inference efficiency caused by the growth of data scale, a cross-model distillation framework based on different teacher and student networks is proposed, and the MLP multi-layer perceptron network optimization structure of the student model replaces the GNN graph neural network optimization structure in the teacher model. Through direct distillation and relational distillation, effective compression of the large model is achieved, thereby achieving higher accuracy and robustness in the process of repairing and recognizing damaged text. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a damaged text recognition method based on a relational distillation graph network according to the present invention; Figure 2 Comparison of time complexity between teacher and student models; Figure 3 Comparison of the space complexity of teacher and student models. DETAILED DESCRIPTION

[0015] The present invention will be further described and illustrated below with reference to the accompanying drawings and specific implementation methods.

[0016] like Figure 1 As shown, a damaged text recognition method based on a relational distillation graph network includes: introducing a radical self-information module based on a teacher model, and performing GNN graph neural network optimization on the node feature vector, so that the teacher model is obtained as having a bipartite graph structure G and a node feature vector X-SIR with radical self-information; Based on the student model, a radical self-information module is introduced, and the node feature vector is optimized by MLP multi-layer perceptron network to obtain the node feature vector X-SIR with radical self-information of the student model; A relational distillation graph network model is constructed, and the student model is distilled and trained using direct distillation and relational distillation. The output probability distribution of the teacher model is transferred to the student model through the distillation process, so that the student model can learn the relational reasoning and structural knowledge of the teacher model. The total loss function in the above distillation process includes the loss function in the teacher model and the loss function for constraining the student during the distillation process.

[0017] This embodiment will introduce the details of each step in detail according to the order of the steps of the above method.

[0018] Teacher Model Introduction: 1. Building Feature Extraction Module: Text Image The image input to the current model is a single text image with a size of 96×96. Resnet18 is used as the backbone network in training to extract image features, as shown in the following formula 1: (1), is the input text image to be recognized, is the output feature vector, It is a combination of a series of convolution, pooling, normalization, nonlinear transformation and other operations based on the ResNet18 model. The total number of component sets and structure sets required to encode Chinese characters.

[0019] 2. Chinese Character Encoding Module: This module breaks down Chinese characters into a hierarchical structure tree based on the Ideographic Description Sequences (IDS) rules of the Unicode 9.0 standard. Based on the position in the structure tree, it calculates the appropriate value for each structure or component node, distinguishes between component and structure nodes, and ultimately constructs an encoding vector for a single character.

[0020] There are 12 spatial structures of Chinese characters, which allows a single structural node to have a maximum of 3 child nodes. Different hyperparameters are set for child nodes at different positions according to the order of child nodes from left to right. When constructing Chinese character encoding according to the hierarchical structure tree, each node has a new attribute, and the number of nodes passed through , which can be considered as the depth of the node in the hierarchical structure tree, starting from the root node, the initial value is 0, and the node impact value is calculated according to the node split depth. , (2), where the hyperparameters The number of nodes passed through The attenuation factor when increasing is 0.5 in the experiment; The depth of each parent node passed through; All parent nodes that the node passes through; To represent the parent node Hyperparameters of . Get the node influence value Then use the hyperparameters Distinguish component nodes and structure nodes, and perform Chinese character encoding as shown below:

[0021] For Chinese character encoding, For a collection of components, is a set of structures, is the component node in the hierarchical structure tree. It is a structural node in the hierarchical structure tree. is the node influence value of the component node, is the one-hot vector of the component, which is 1 in the dimension representing the corresponding component. is the one-hot vector of the structure, which is 1 in the dimension representing the corresponding structure, where the hyperparameter The value of is 0.5.

[0022] 3. Constructing a bidirectional embedding module: In order to reduce the information loss caused by spatial mapping, the bidirectional embedding migration module not only realizes the mapping from feature space to encoding space, but also realizes the mapping from encoding space to feature space, that is, bidirectional embedding to constrain the migration and transformation of information. This module is composed of a fully connected layer. In order to ensure that there will be no negative values, ReLU is used as the activation function to transform the output of the feature extraction module. Input into the module and then multiplied by the trainable parameters , applied to the ReLU function, as shown in Formula 4: (4) is a trainable parameter used to map the feature space to the encoding space, is the feature space vector input to the bidirectional embedding transfer module, The Chinese character encoding output by the bidirectional embedding migration module; The bidirectional migration module also has the ability to restore the encoding space to the feature space, as shown in Formula 5, and also uses the ReLU function. (5), is a trainable parameter, The transpose of is the recovered feature vector; in order to measure the encoding obtained after mapping from the feature space to the encoding space The real Chinese character code corresponding to the text to be recognized The difference between them, the recovery loss function is designed as shown in Formula 6, (6) Among them, The encoding of the Chinese character class corresponding to the text to be recognized; The total number of component sets and structure sets required for encoding Chinese characters is 405.

[0023] At the same time, in order to evaluate the similarity between the vector restored from the encoding space to the feature space and the original input feature space vector, the reconstruction loss is designed as shown in Formula 7: (7) Among them, is the dimension of the vector in the feature space.

[0024] 4. Constructing the damage judgment module: feature vector At the same time, it also needs to be sent to the damage judgment module to determine whether the character is complete or incomplete. The damage judgment module consists of only one fully connected layer. Get the final output probability and determine the damage of Chinese characters, as shown in formula 8. (8) in, is a trainable parameter, and its output is a binary classification vector , its loss function The design uses the cross entropy function, (9) represents the cross entropy function, Indicates the actual damage condition of the characters.

[0025] 5. Component retrieval strategy: The retrieval input of the character candidate set is the components extracted from the image, which is the component category set , and Chinese character encoding set , visualize the Chinese character encoding set, components correspond to table columns, Chinese characters correspond to table rows, for component category set A separate component in ,use The corresponding component column filters out all Chinese character rows corresponding to non-zero cells in the component column to form the character candidate subset of the component After filtering all the components in the component category set, we get the character candidate subset set, and finally find the intersection of the character candidate subset set to get the character candidate set , as shown in formula (10), (10) To utilize the The character candidate subset obtained after searching the Chinese character code set, For component category sets The number of internal components, If the damage module determines that the text is damaged, it needs to use the character candidate set to search on the Chinese character encoding set, where the components correspond to the table columns and the Chinese characters correspond to the table rows. A single character in ,use The corresponding Chinese character row, filter out the component columns corresponding to the non-zero cells in the Chinese character row, and form the component candidate subset of the character After all the Chinese characters in the character candidate set are screened, the component candidate subset set is obtained. Finally, the union of all the component candidate subset sets is obtained to obtain the component candidate set. (11) To utilize the The component candidate subset obtained after searching the Chinese character code set, Character candidate set The number of characters in the string.

[0026] 6. Introducing the radical self-information module: The step of introducing the radical self-information module based on the teacher model includes: Input, first count the components or structures The probability of appearing in 2527 first-level Chinese characters, and then calculating the radical self-information, is shown in the following formula: (12) in, For components or structures The amount of self-information, For components or structures Probability of appearing in 2527 characters; After obtaining the radical self-information, the "1" in the one-hot initial feature vector of the original component node in the bipartite graph structure is replaced with the corresponding self-information value, and the teacher model is obtained as having a bipartite graph structure G and a node feature vector X-SIR with radical self-information.

[0027] 7. Bipartite graph module construction: will go through HDE [5] The text encoding obtained after encoding , Chinese character encoding set , and the component candidate set Input into the bipartite graph module. Suppose bipartite graph ,in Represents a collection of nodes, represents the edge set, Represents a node in the graph The representation of Represents a node in the graph and nodes The weighted edges of .

[0028] The bipartite graph has been added with the information of the text to be recognized before being sent to the graph network, so the node representation of the input in layer 0 is The node set represented It consists of two parts, one of which is the Chinese character encoding set. The character node set consisting of Chinese characters and characters to be recognized , the other part is the node set composed of component nodes and structure nodes , the same edge embedding It also consists of two parts, one of which is the HDE code A non-zero value , part of which is the weighted edge connecting the text node to be recognized and the component node .

[0029] First, the input node representation and edge embedding are concatenated as shown in Equation 13: (13) in, is the feature splicing operation, for the layer, the input is Node representation of layer output and edge embedding .

[0030] As input, a linear transformation is performed, and the model can adjust the parameters based on the training data to mine the potential feature patterns in the data: (14) is the trainable parameter matrix.

[0031] Then Apply to the activation function to get the intermediate variable

[0032] (15) is the activation function.

[0033] On the other hand, for nodes All adjacent nodes of , As the node neighborhood, we will get a series of , combining these results into a set that contains the nodes Intermediate feature information after interacting with all adjacent nodes: (16) Finally, through the aggregation function, Aggregate the elements in to get the node In the The final feature representation of the layer , as shown in the following formula (17): (17) in, is an aggregate function, It represents the node after aggregation.

[0034] Each time a different text is recognized, the input bipartite graph data will change according to the text to be recognized, and after updating, a new node embedding is obtained. , as shown in the following formula, first represent the aggregated nodes Hedi Node representation of the layer Perform feature stitching: (18) in, It is a feature splicing operation.

[0035] Then Through trainable parameters The original feature space is mapped to a new feature space, and then applied to the activation function to introduce nonlinear transformation to improve the generalization ability of the model, as shown in the following formula: (19) is a trainable parameter, Embed the new node.

[0036] Finally, the edge embedding is updated to obtain , similarly, first embed the updated node into the The edge embeddings of the layer are aggregated and applied to the activation function after multiplying by the trainable parameters, as shown in the following formula: (20), where For splicing operation, is a trainable parameter, For the edge Node characterization, For the edge Node characterization, is the activation function.

[0037] Now get the Node representation of layer output and edge embedding , the graph network optimizes the representation of character nodes and components or structural nodes, learns the mapping relationship between Chinese characters and components, and lays a good foundation for downstream edge prediction to establish a mapping from damaged characters to components.

[0038] 8. Edge prediction module construction: The above optimized bipartite graph network node representation The input is sent to the edge prediction module, which is composed of a multi-layer perceptron with three fully connected layers. The activation function is the ReLU function. When passing through the first layer, the linear layer dimension is 512, and the probability that each neuron may be discarded in the back propagation is 0.5. Then after passing through the second layer, the dimension becomes 256, and after passing through the third layer, the linear layer dimension becomes 1. Finally, the new relationship between the text nodes and components to be identified and the structure nodes is obtained. , used to reconstruct Chinese character encoding and Chinese character recognition, as shown below: (twenty one) Represents a splicing operation, is the multi-layer perceptron MLP for edge prediction, For the edge Character node representation, For the edge Component or structural node representation, is the prediction result of a specific weighted edge.

[0039] After completing the reconstruction of the edge relationship between the character nodes and components to be recognized and the structural nodes, it can be used to construct the loss function on the one hand, and on the other hand, for character recognition, as shown in the following formula 22: (twenty two) Encode the value of a specific dimension for a text, The total number of component sets and structure sets required for encoding Chinese characters is 405.

[0040] 9. Metric Identification: In character recognition, we first use the new edge relationship between the text nodes and components to be recognized and the structure nodes Construct the corresponding encoding of the text to be recognized , and then measure the Chinese character encoding corresponding to the character in the character candidate set by the cosine distance, as shown in the following formula, = (twenty three) is the code of the Chinese character to be recognized, is a set of Chinese character codes corresponding to the characters in the character candidate set. is the value obtained by calculating the cosine distance. The above is the complete module content of the teacher model.

[0041] The student model's structure is largely similar to that of the teacher model, differing in that the GNN network, which optimizes node feature vectors, is replaced with a more lightweight multi-layer perceptron (MLP) network. This network uses only the initial node features as input, independently of the graph structure, and directly performs feature transformation to optimize the node's features. Because the MLP network lacks graph structure, it misses the inclusion relationship between components and structures in Chinese characters, making direct training of the student model for damaged text restoration impractical. Therefore, a distillation method is employed to transfer key knowledge from the teacher GNN to the student model during training, allowing the student model to gain additional knowledge of the graph structure.

[0042] A relational distillation graph network model is constructed. The teacher model takes as input the bipartite graph structure G and the node feature vector X-SIR with radical self-information, while the student model takes as input only the node feature vector X-SIR with radical self-information. The model uses two training methods: direct distillation and relational distillation.

[0043] The direct distillation step comprises: first The representation output of the second layer and student models Node representation output generated by multi-layer perceptron network optimization The mean squared error loss is calculated to measure the similarity between the student model and the teacher model in terms of node features, as shown in the following formula: (twenty four) in, Represents all nodes in the graph structure, Indicates the use of mean square error loss, i Represents a component; The node feature vector input by the teacher model is then optimized by the GNN graph neural network. The edge prediction module is used to obtain the new edge relationship D ̂uv between the text node to be identified and the component and structure node on the optimized vector node representation, thereby constructing the corresponding encoding (hde) ̅_char of the text to be identified and obtaining the loss function of the logical matching. : (25) in, Indicates the set that needs to be filled in the edge prediction module; represents the edge weight predicted by the edge prediction module in the teacher model; Represents the edge weight predicted by the edge prediction module in the student model; To use mean square error loss, i Indicates a component, j Representing the graph structure; finally, aligning the outputs of the teacher model and the student model in the edge prediction module, so that the student model can learn the relational reasoning and structural knowledge of the teacher model.

[0044] Relation Distillation Method: We adopt two relation distillation strategies, ranking matching and distribution matching, to rank or reason about relations of target nodes.

[0045] First, establish the target text node The topological structure of the central node is used to uniformly map the set of text nodes to be recognized to the central node. The context node collection A dynamically scalable graph structure connects the components that require edge prediction. Before performing relationship distillation, the edge weights output by the edge prediction module are processed, retaining the original predicted edge weights for downstream tasks such as filling in text encoding and performing damaged text recognition.

[0046] The relationship distillation steps include: introducing an independent "radical relevance score" , the edge weights output by the edge prediction module are subjected to probability normalization and converted into standardized values ​​representing the predicted probability of connections between nodes. Specifically: the edge weights of the teacher model are obtained by Function normalization yields , as a probabilistic representation of its radical association; the student model uses the same method to generate the corresponding score , both are jointly optimized through sorting and distribution consistency in the distillation stage.

[0047] As shown in formulas 26 and 27: (26) represents the edge weight predicted by the edge prediction module in the teacher model; Represents the text node to be recognized in the teacher model The set of components that need edge prediction; (27) represents the edge weight predicted by the edge prediction module in the student model; Represents the text node to be recognized in the student model The set of components that need edge prediction.

[0048] At the same time, a teacher model uses ranking matching to rank a set of possible radical candidates. This ranking relationship is then distilled to the student model, allowing it to learn the relative importance of these radicals. We use a margin-based ranking loss function to transform this ranking-based approach into a training objective, using the teacher's output to predict the ranking to train the student model.

[0049] The step of relation distillation further includes: calculating the predicted probability of the student model output The difference between the two context nodes predicted by the student model is combined with the information of the teacher model to determine the central node 𝑣. and Who is responsible for the node The relative ordering relationship between point pairs enables the student model to learn the relationship knowledge between nodes.

[0050] As shown in formula 28, (28) Where V represents the set of all central nodes, that is, the set of damaged characters that need to be recognized; Indicates the hyperparameter, which is 0.2 in the experiment. It represents the knowledge transferred by the teacher and is defined as shown in Equation 29: (29) Boundary hyperparameters It is used to prevent the student model from being overly sensitive to small probability differences in the teacher model that may be caused by noise.

[0051] Then we further use a distribution-based matching loss function To take full advantage of teachers Specifically, using the teacher's prediction and student predictions Perform KL divergence calculation to let the student model match the relative value within the probability distribution of the context nodes based on each central node, supplementing the numerical difference information lacking in ranking matching, as shown in the following formula: (30) Where V represents the set of all central nodes, that is, the set of damaged characters that need to be recognized; It is a temperature hyperparameter, which is set to 0.8 in the experiment to control the smoothness of the probability distribution.

[0052] Loss function introduction: The graph network lossy text recognition model GNN2MLP based on relational distillation adopts a multi-task joint optimization strategy. Its total loss consists of two parts: one is the loss function in the teacher model, and the other is the loss function that constrains the student during the distillation process. It is consistent with the loss function defined in DynamicGrape, as shown in Equation 31: (31) Wherein is the binary classification loss of the damage judgment module, is the mapping loss of the bidirectional embedding transfer module, is the reconstruction loss of the bidirectional embedding transfer module, is the prediction loss of the weighted edge prediction module, , , , are the weight hyperparameters of each loss item. In the experiment, the weight hyperparameter value of each loss item is 1.

[0053] The loss function that constrains the student during the distillation process It is composed of the losses of four matching strategies as follows: (32), where To represent the matching loss, is the logistic matching loss, is the ranking matching loss, is the distribution matching loss, , , , Different weights are assigned to each loss. In the experiment, the weight value of each loss is 1.

[0054] Model GNN2MLP total loss function As shown in the following formula: (33) Where, , are the weight hyperparameters of each loss item, and their experimental median value is 1.

[0055] In order to verify the performance of the proposed model, comparative experiments were conducted on the component-coding dataset and the CTW dataset to evaluate the recognition performance of each method in the face of missing characters, incomplete shapes, etc., thereby verifying the performance advantages of the present invention in the damaged character scenario, as shown in Tables 1 and 2.

[0056] Table 1 Comparative experiment of text recognition methods under the component-code dataset

[0057] Table 2 Comparative experiments on text recognition methods using the CTW dataset

[0058] Experimental results show that compared with other methods, the method of the present invention achieves the best recognition accuracy in most evaluation indicators. Traditional image classification-based neural network models, such as CRNN, ResNet, and DenseNet, are not designed for zero-shot character recognition and therefore cannot work effectively in component-code datasets. Their accuracy is significantly lower than that of other zero-shot methods, and some methods may fail completely. RCN, ZCTRN, CCDF, and PBS do not consider damage recovery and use distance for recognition. They face problems including overly dense model prediction codes and an excessive number of character class candidates in distance calculations, resulting in the failure of RCN, CCDF, and PBS and low recognition accuracy of ZCTRN. Relatively good results were achieved on the CTW dataset, demonstrating higher accuracy, verifying its applicability and stability in real, non-damaged text scenarios.

[0059] In order to measure the effectiveness of model compression, this paper compares the time complexity and complexity of the GNN module in the teacher model and the MLP module in the student model. Figure 2 Shown is a line graph comparing time complexity and space complexity. Figure 2 The figure shows the time complexity. Figure 3 The figure shows space complexity, with the horizontal axis representing the number of nodes and the vertical axis representing the number of floating-point operations. As can be seen from the figure, GNNs significantly outperform MLPs in both time and space complexity, with the difference being particularly pronounced when processing large-scale graph data. This also indirectly demonstrates the advantages of MLPs as student models in terms of inference efficiency and resource consumption.

Claims

1. A damaged text recognition method based on a relational distillation graph network, characterized in that: The following steps are involved: Based on the teacher model, a radical self-information module is introduced, and the node feature vector is optimized by GNN graph neural network, so that the teacher model has a bipartite graph structure G and a node feature vector X-SIR with radical self-information; Based on the student model, a radical self-information module is introduced, and the node feature vector is optimized by MLP multi-layer perceptron network to obtain the node feature vector X-SIR with radical self-information of the student model; A relational distillation graph network model is constructed, and the student model is distilled and trained using direct distillation and relational distillation. The output probability distribution of the teacher model is transferred to the student model through the distillation process, so that the student model can learn the relational reasoning and structural knowledge of the teacher model. The total loss function in the above distillation process includes the loss function in the teacher model and the loss function for constraining the student during the distillation process.

2. The damaged text recognition method based on the relational distillation graph network according to claim 1, characterized in that: The step of introducing the radical self-information module based on the teacher model includes: Input, first count the components or structures The probability of appearing in 2527 first-level Chinese characters, and then calculating the radical self-information, is shown in the following formula: ,in, For components or structures The amount of self-information, For components or structures Probability of appearing in 2527 characters; After obtaining the radical self-information, the "1" in the one-hot initial feature vector of the original component node in the bipartite graph structure is replaced with the corresponding self-information value, and the teacher model is obtained as having the bipartite graph structure G and the node feature vector X-SIR with the radical self-information.

3. The damaged text recognition method based on the relational distillation graph network according to claim 1, characterized in that: The direct distillation step comprises: first The representation output of the second layer and student models Node representation output generated by multi-layer perceptron network optimization The mean squared error loss is calculated to measure the similarity between the student model and the teacher model in terms of node features, as shown in the following formula: ,in, Represents all nodes in the graph structure, Indicates the use of mean square error loss, i Indicates a component or word structure; The node feature vector input by the teacher model is then optimized by the GNN graph neural network. The edge prediction module is used to obtain the new edge relationship D ̂uv between the text node to be identified and the component and structure node on the optimized vector node representation, thereby constructing the corresponding encoding (hde) ̅_char of the text to be identified and obtaining the loss function of the logical matching. : ,in, Indicates the set that needs to be filled in the edge prediction module; represents the edge weight predicted by the edge prediction module in the teacher model; represents the edge weight predicted by the edge prediction module in the student model; To use mean square error loss, i, j Indicates a component or word structure; Finally, the outputs of the teacher model and the student model in the edge prediction module are aligned, so that the student model can learn the relational reasoning and structural knowledge of the teacher model.

4. The damaged text recognition method based on the relational distillation graph network according to claim 1, characterized in that: The relationship distillation step includes: introducing an independent radical relevance score , the edge weights output by the edge prediction module are subjected to probability normalization and converted into standardized values ​​representing the predicted probability of connections between nodes. Specifically: the edge weights of the teacher model are obtained by Function normalization yields , as a probabilistic representation of its radical association; the student model uses the same method to generate the corresponding score , both are jointly optimized through sorting and distribution consistency in the distillation stage.

5. The damaged text recognition method based on the relational distillation graph network according to claim 4 is characterized in that: The step of relation distillation further includes: calculating the predicted probability of the student model output The difference between the two context nodes predicted by the student model is combined with the information of the teacher model to determine the central node 𝑣. and Who is responsible for the node The relative ordering relationship between point pairs enables the student model to learn the relationship knowledge between nodes.

6. The damaged text recognition method based on the relational distillation graph network according to claim 1, characterized in that: The loss function in the teacher model As shown in the following formula: in, is the binary classification loss of the damage judgment module, is the mapping loss of the bidirectional embedding transfer module, is the reconstruction loss of the bidirectional embedding transfer module, is the prediction loss of the weighted edge prediction module, , , , are the weight hyperparameters of each loss term.

7. The damaged text recognition method based on the relational distillation graph network according to claim 1, characterized in that: The loss function that constrains the student during the distillation process It is composed of the losses of four matching strategies as follows: Where, To represent the matching loss, is the logistic matching loss, is the ranking matching loss, is the distribution matching loss, , , , Different weights are assigned to each loss.

Images