Chinese character recognition method based on Chinese character similarity algorithm joint optimization

Abstract

The invention discloses a Chinese character recognition method based on Chinese character similarity algorithm joint optimization. Comprising the steps of obtaining a to-be-recognized Chinese character image, inputting the preprocessed image into a convolutional neural network classifier, and outputting a Chinese character category prediction result; obtaining a corresponding standard Chinese character image from the Chinese character database, and respectively calculating stroke similarity, structural similarity and topological similarity between the to-be-recognized image and the standard image; calculating a Chinese character similarity score based on multiple similarity fusion; and constructing a joint loss function containing classification loss and similarity loss, and carrying out joint training on the classifier network. By introducing a stroke template feature extraction and dynamic time warping algorithm, accurate matching of stroke forms and sequences is realized. The technical problem that similar Chinese characters are difficult to distinguish in a traditional recognition method is effectively solved, deep fusion and collaborative optimization of classification and similarity information are achieved, the accuracy and stability of Chinese character recognition are remarkably improved, and the method is suitable for diversified application scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of Chinese character recognition, and in particular relates to a Chinese character recognition method based on joint optimization of a Chinese character similarity algorithm. Background Art

[0002] Chinese character recognition is a key research area in pattern recognition and computer vision, widely used in document digitization, image retrieval, human-computer interaction, and other practical scenarios. With the development of deep learning, models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been widely introduced into Chinese character recognition tasks, significantly improving recognition accuracy and efficiency.

[0003] Most current mainstream Chinese character recognition methods employ an end-to-end architecture based on image classification, directly mapping input images to predefined Chinese character categories by training neural network models. These methods perform well with standard fonts and clear images. However, due to the large number of strokes, complex structure, diverse radical combinations, and the large number of similar characters, traditional classification models experience a significant decline in recognition accuracy when dealing with writing variants, font changes, or noisy images.

[0004] Prior art, Chinese patent CN117636372A discloses a Chinese character recognition method, apparatus, computer device, and storage medium. The method comprises: obtaining an image of a Chinese character to be recognized; inputting the image into a preset recognition network, and using the recognition network to recognize the Chinese character in the image to be recognized, thereby determining the category of the Chinese character to be recognized; wherein the recognition network is trained using sample images and an adaptive margin loss function, and different categories of Chinese characters have different weights in the adaptive margin loss function.

[0005] However, this method still has some limitations. On the one hand, the recognition process mainly relies on the convolutional neural network to extract the overall features of the image and the discriminant ability of the classifier, and lacks the modeling and analysis of the internal structural information of Chinese characters (such as stroke order, structural components, spatial layout, etc.). On the other hand, although an adaptive margin loss function has been introduced to improve the recognition effect of similar characters, it is essentially still based on the matching of image-level global features and lacks the ability to perceive subtle structural differences. Therefore, when dealing with Chinese character images with varying writing styles, similar stroke details, or large character deformations, it is still difficult to achieve accurate recognition.

[0006] Furthermore, most current methods use the similarity measurement algorithm and the classifier module independently, preventing them from sharing feature information and limiting the model's ability to discern subtle differences. This structure often suffers from errors and insufficient utilization of local features, making it prone to recognition errors, especially when dealing with highly similar Chinese characters. As a result, the overall recognition system's accuracy in identifying similar characters still needs to be improved. Summary of the Invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] The present invention provides a Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm, comprising the following steps:

[0010] Obtain the Chinese character image to be recognized and perform preprocessing;

[0011] Inputting the preprocessed image into a Chinese character classifier network and outputting a corresponding Chinese character category prediction result;

[0012] Obtaining a standard Chinese character image corresponding to the Chinese character category prediction result from the constructed Chinese character image database;

[0013] respectively calculating the stroke similarity, structural similarity and topological similarity between the Chinese character image to be recognized and the standard Chinese character image;

[0014] Calculating a Chinese character similarity score between the Chinese character image to be recognized and the standard Chinese character image based on the fusion of the stroke similarity, structural similarity and topological similarity;

[0015] A joint loss function including classification loss and similarity loss is constructed based on the Chinese character category prediction results and the Chinese character similarity scores;

[0016] The Chinese character classifier network is back-propagated and trained based on the joint loss function to obtain the trained Chinese character classifier network, thereby realizing the recognition of Chinese characters.

[0017] Furthermore, the preprocessing includes binarization, normalization and grayscale processing of the image to obtain a standard image input of uniform size.

[0018] Furthermore, the Chinese character classifier network is an image classification model based on a convolutional neural network (CNN), specifically including multiple convolutional layers, pooling layers, normalization layers and fully connected layers.

[0019] Furthermore, the stroke similarity calculation formula is:

[0020]

[0021] Among them, S stroke is the stroke similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, a is the weight coefficient, γ is a positive real number adjustment parameter, Sh A 、ShB are the pen shape template features of the Chinese character image A to be recognized and the standard Chinese character image B, SSIM (Sh A ,Sh B ) is the cosine similarity of the pen shape template features of the Chinese character image A to be recognized and the standard Chinese character image B, DTW(sp A ,sp B ) is the dynamic time warping distance, which is used to measure the similarity between the sequence sequences of two images.

[0022] Furthermore, the process of acquiring the features of the pen-shaped template includes:

[0023] Grayscale the Chinese character image and convert it into a grayscale image;

[0024] Normalize the grayscale image and adjust it to a uniform size of 32×32 pixels;

[0025] The gradient of the normalized 32×32 grayscale image is calculated, and the horizontal gradient G is calculated using the Sobel operator. x With the vertical gradient G y , specifically:

[0026] G x =I*S x ,G y =I*S y

[0027] Among them, I is the normalized grayscale image, S x 、S y They are horizontal and vertical Sobel filter kernels respectively, and * is the convolution operation;

[0028] Based on the horizontal gradient G x With the vertical gradient G y Calculate the gradient magnitude G(x,y) and gradient direction θ(x,y) of each pixel point. The expression is:

[0029]

[0030] Where G(x,y) is the gradient magnitude of the pixel point (x,y), and θ(x,y) is the gradient direction of the pixel point (x,y).

[0031] The image is divided into several 2×2 pixel cell units. For each pixel point in the cell, the gradient amplitude G(x,y) is projected to the corresponding interval of the predefined direction histogram according to its gradient direction, and the gradient direction histogram is statistically formed.

[0032] Multiple adjacent cell units are grouped into a block, and the gradient direction histogram of the block is normalized by L2-Hys to eliminate the influence of illumination and contrast. The normalization formula is:

[0033]

[0034] Among them, v norm is the normalized gradient histogram vector, v is the gradient histogram vector within the block, and ∈ is a stable constant to prevent the denominator from being zero;

[0035] The normalized gradient direction histogram vectors of all blocks in the image are spliced in sequence to form the final pen-shaped template feature vector S h .

[0036] Furthermore, the dynamic time warping distance is:

[0037]

[0038] Among them, DTW(sp A ,sp B ) represents the dynamic time warping distance between the stroke sequence of the Chinese character image A to be recognized and the standard Chinese character image B, sp A (i) and sp B (π(i)) represents the i-th stroke vector of the Chinese character image A to be recognized and the corresponding π(i)-th stroke vector of the standard Chinese character image B on the alignment path π, where the stroke vector is obtained by extracting the start and end coordinates, direction, length and stroke morphological features of each stroke after segmenting the Chinese character image; ‖‖2 is the Euclidean norm, π is the alignment path, and the optimal alignment path is obtained by the dynamic time warping DTW algorithm; n = min(|sp A |,|sp B |) is the length of the shorter stroke sequence, |sp A | is the number of strokes of the Chinese character image A to be recognized, π(i) is the number of strokes sp in the path π A (i) Aligned sp B The stroke index in .

[0039] Furthermore, the structural similarity calculation process is:

[0040] For the Chinese character images to be recognized and the standard Chinese character images, a character segmentation algorithm based on OpenCV is used to perform structural decomposition with the maximum decomposition degree. Each Chinese character is decomposed into several sub-components. Based on the decomposition results, a hierarchical structure tree is constructed. Each layer of the structure tree represents the decomposition level of the Chinese character. The root node corresponds to the complete Chinese character, and the leaf node corresponds to the basic component. In the structure tree, each node represents the sub-component features and spatial position of the corresponding level. The child nodes of the node are the further decomposed components, forming a recursive tree structure.

[0041] Determine whether the structures of two Chinese characters are the same, provided that the topological structures of the structure trees are the same and the number and categories of subcomponents at the corresponding levels match; if this condition is not met, the structures are determined to be different;

[0042] When the structure is the same, calculate the weight attenuation coefficient ω of each level sub-component in the structure tree k The calculation formula is:

[0043]

[0044] Among them, ω j is the weight attenuation coefficient of the kth layer; K is the maximum decomposition level of the structure tree;

[0045] Based on the structure tree matching algorithm, the best matching mapping path σ of the structural component is obtained, which is used to map the kth subcomponent A of the Chinese character to be recognized. j Corresponding subcomponent B of standard Chinese characters σ(k) Perform alignment and calculate the structural alignment score based on the weight decay coefficient:

[0046]

[0047] Among them, align score is the structural alignment score, sim(A k ,B σ(k) ) represents the HOG feature similarity between the k-th sub-components;

[0048] Calculate the structural similarity score based on the structural alignment score:

[0049]

[0050] Among them, S struct is the structural similarity between the Chinese character image A to be recognized and the standard Chinese character image B, |A| and |B| are the total number of subcomponents of the Chinese character image A to be recognized and the standard Chinese character image B, b is the penalty coefficient for position difference, Δ pos To match the position differences of sub-components, the Euclidean distance of the corresponding sub-component center coordinates is calculated, specifically:

[0051]

[0052] in, and Subcomponent A k and B σ(k) The center coordinates of

[0053] When the structures are different, the Jaccard similarity of the radical set is calculated as the structural similarity:

[0054]

[0055] Among them, C A 、C B are the radical sets of the Chinese character image A to be recognized and the standard Chinese character image B, respectively, where |C A ∩C B | represents the number of radicals shared by the two images, |C A ∪C B | is the union number of the radicals of the two images.

[0056] Furthermore, the topological similarity calculation process includes:

[0057] Binarize and normalize the Chinese character image A to be recognized and the standard Chinese character image B to obtain binary images of uniform size;

[0058] The Zhang-Suen thinning algorithm is applied to the binary image for skeleton extraction. The Zhang-Suen algorithm is based on pixel neighborhood iterative processing. Through a two-stage iterative rule, edge pixels are gradually removed while maintaining image connectivity and structural integrity, ultimately retaining a single-pixel width skeleton image of the stroke trunk;

[0059] Perform pixel connectivity analysis on the generated skeleton graph, extract key pixels such as intersections and endpoints, and construct the topological graph structure G = (V, E) of the stroke skeleton, where: V = {v1, v2, ..., v m} is a node set, representing the key points in the stroke skeleton graph, including intersections, endpoints, and turning points. The nodes are extracted from the skeleton graph through pixel connectivity detection; E = {e ij ∣(v i ,v j )∈V×V} is an edge set, which represents the connection relationship between nodes formed by stroke paths; each edge e ij Including the eigenvector f ij , including the length, direction, and curvature of the edge, which is used for subsequent weighted matching;

[0060] Constructing graph structure G A =(V A ,E A ),GB =(V B ,E B ), respectively representing the stroke skeleton diagrams of the Chinese character image A to be recognized and the standard Chinese character image B;

[0061] The maximum weight matching MWM algorithm is used to A =(V A ,E A ),G B =(V B ,E B ) for structural matching, for any pair of edges and The similarity weight is defined as:

[0062]

[0063] in, Represents the graph structure G respectively A =(V A ,E A ),G B =(V B ,E B ) in the two stroke path edges, represents the Euclidean distance between the eigenvectors of two edges, and β is a positive control coefficient;

[0064] Construct a complete bipartite graph G M =(E A ,E B ,W), perform the maximum weight matching MWM algorithm on it to obtain the optimal matching set, and calculate the maximum value of the total weight in the complete bipartite graph matching:

[0065]

[0066] Among them, M is the edge matching set, MWM(G A ,G B ) is the graph structure G A =(V A ,E A ),G B =(V B ,E B ) The total similarity score under maximum matching;

[0067] Calculate the topological similarity based on the total similarity score under the maximum matching:

[0068]

[0069] Among them, ‖E A ‖,‖E B ‖ respectively represent the graph structure GA =(V A ,E A ), G B =(V B ,E B ) is the number of edges.

[0070] Furthermore, the Chinese character similarity score formula is:

[0071] sim(A, B)=c·S stroke +d·S struct +(1-cd)·S topo

[0072] Among them, sim(A, B) is the Chinese character similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, S stroke 、S struct 、S topo are the stroke similarity, structural similarity and topological similarity between the Chinese character image A to be recognized and the standard Chinese character image B, respectively. c and d are weight coefficients.

[0073] Furthermore, the joint loss function is:

[0074]

[0075] in, is the joint loss function, C is the total number of Chinese character categories, y i Represents the true category label of the i-th Chinese character category. When the Chinese character image to be recognized is the i-th Chinese character category, y i =1, the rest are 0, is the probability of the i-th category predicted by the classifier, sim(A, B) is the Chinese character similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, and λ1 and λ2 are weight coefficients.

[0076] Compared with the prior art, the present invention has the following advantages:

[0077] (1) The present invention solves the problem in the prior art that Chinese character recognition relies on a single global classification feature and is difficult to accurately distinguish Chinese characters with similar morphology and complex structure through a multi-dimensional Chinese character similarity calculation method based on stroke similarity, structural similarity and topological similarity. It realizes comprehensive perception and accurate measurement of Chinese character stroke details, component structure and stroke topological relationship, and significantly improves the recognition accuracy and robustness of similar characters.

[0078] (2) The present invention solves the technical problem of the separation of the classifier and the similarity module in the traditional recognition system and the inability to share and jointly optimize feature information by integrating the joint loss function training mechanism of the Chinese character classifier network with the similarity. It realizes the collaborative learning and optimization of classification information and similarity information, improves the classifier's sensitivity and discrimination ability to subtle differences, and enhances the generalization performance and stability of the recognition model.

[0079] (3) The present invention solves the problem of recognition error caused by large changes in stroke shape and stroke order through the extraction of stroke template features and dynamic time warping (DTW) distance measurement technology in stroke similarity calculation, realizes accurate matching of stroke shape and dynamic changes in stroke order, and effectively improves the fault tolerance for the diversity of writing styles and differences in stroke order.

[0080] (4) The present invention solves the problem of the influence of input image size and illumination difference on feature stability through grayscale conversion, normalization processing and gradient directional histogram (HOG) feature extraction method, realizes the expression of pen shape features that are unifying scale and independent of illumination, improves the robustness and consistency of feature extraction, and ensures the adaptability of the recognition system to diversified input images. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 is a flow chart of the method of the present invention;

[0082] Figure 2 Flowchart of the Chinese character similarity calculation method of the present invention;

[0083] Figure 3 This is a flow chart of the stroke similarity calculation method of the present invention;

[0084] Figure 4 This is a flow chart of the structural similarity calculation method of the present invention;

[0085] Figure 5 This is a flow chart of the topology similarity calculation method of the present invention;

[0086] Figure 6 Schematic diagram of the gradient direction histogram of the present invention;

[0087] Figure 7 It is a structural diagram of the present invention. DETAILED DESCRIPTION

[0088] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0089] This embodiment provides a Chinese character recognition method jointly optimized based on a Chinese character similarity algorithm, as Figure 1 shown, including the following steps:

[0090] Step S1: Obtain the Chinese character image to be recognized and perform preprocessing, which includes binaryzation, normalization, and grayscale processing of the image to obtain a standard image input of a unified size.

[0091] Step S2: Input the preprocessed image into a Chinese character classifier network and output the corresponding predicted result of the Chinese character category; the Chinese character classifier network is an image classification model based on the convolutional neural network CNN, specifically including multiple convolutional layers, pooling layers, normalization layers, and fully connected layers.

[0092] Step S3: Obtain the standard Chinese character image corresponding to the predicted result of the Chinese character category from the constructed Chinese character image database, where the Chinese character image database includes a set of standard glyph images constructed by manual or rule sampling, the image set covers common Chinese character characters, and provides standard writing images with clear structures and complete strokes for each Chinese character category;

[0093] Step S4: Calculate the stroke similarity, structure similarity, and topological similarity between the Chinese character image to be recognized and the standard Chinese character image respectively;

[0094] The calculation of Chinese character similarity needs to consider the following factors:

[0095] (1) Similarity of single characters (components): The number of single characters (components) is not large. Some are quite different, and some are very similar and are easily misrecognized. We classify and group them according to the degree of external similarity. The first level is basically the same, such as '日' '口', '七' '匕'; the second level is different but very similar, such as '干' '丁' '千', and '人' is not similar to each other.

[0096] (2) Similarity of compound character structures: Generally speaking, compound characters are not similar to single characters, and compound characters with different first-level structures are not similar either. Two compound characters with the same first-level structure have a basis for similarity, and the final similarity also depends on the similarity of the two pairs of corresponding components. The deeper the structure, the smaller the impact of its similarity on the overall similarity. For example, the similarity between '椅' and '倚、猗、椅' is very high because they are all left-right structures and the right sides are exactly the same and the left sides are very similar. The similarity between '椅' and '琦' is slightly worse because the difference between the left wood radical and the oblique king radical in them is relatively large. The similarity between '椅' and '掩' is even worse because their left parts are different, and although the right parts are both upper-lower structures to support the similarity, the difference between '可' and '也' in the lower right part is very large. Therefore, we designed a recursive algorithm for comparing the similarity of compound characters according to the structural level.

[0097] There are exceptions to the above situation. For example, "真", "其", and "具" are all compound characters with an upper and lower structure, but they are basically similar in shape to the single-component character "其". According to the degree of similarity in shape, they should be classified as third-level similar characters. Another example is that "血" and "四" are both compound characters, with "血" having an upper and lower structure and "四" having an embedded structure. Although their structures are different, from the perspective of perception, their shapes are basically similar, and they should also be classified as third-level similar characters. For these special compound characters, we treat them in the same way as single-component characters and list them together with similar single-component characters in the similarity character table.

[0098] Another example is that "勺", "句", "勾", and "司" are both third-level similar characters and are all compound characters with a right-upper enclosure (the first-level structure is the same). Similar cases include "左", "右", and "石", which are both second-level similar characters and are all compound characters with a left-upper enclosure. Finally, the similarity value should be taken as the larger of the two calculation results obtained by comparing the similarity level and the structural level.

[0099] (3) The influence of strokes on similarity: For two single-component characters that are not in the similarity character table or one single-component character and one compound character, as well as compound characters that are not in the similarity character table and have different structures, although they are completely dissimilar, the degree of their dissimilarity also needs to be quantitatively given because they may be components of two other similar characters. Generally speaking, the difference in the number of strokes of the two characters can be used to measure their degree of dissimilarity. For example, the right part of "椅" is very dissimilar to that of "朴", not only because one is a compound character and the other is a single-component character, but also because the difference in the number of strokes of these two components is quite large.

[0100] Based on the above three considerations, the designed glyph similarity algorithm adopts a multi-layer recursive decomposition strategy, integrating stroke features, structural features, and topological features, and defines the process of the Chinese character similarity algorithm as Figure 2 shown below.

[0101] Among them, as Figure 3 shown below, the process of calculating stroke similarity is as follows:

[0102]

[0103] Among them, S stroke is the stroke similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, a is the weight coefficient, γ is the adjustment parameter of positive real numbers, Sh A , Sh B are respectively the pen shape template features of the Chinese character image A to be recognized and the standard Chinese character image B, SSIM(Sh A , Sh B ) is the cosine similarity of the pen shape template features of the Chinese character image A to be recognized and the standard Chinese character image B, DTW(sp A , spB ) is the dynamic time warping distance, which is used to measure the similarity between the sequence sequences of two images.

[0104] The process of acquiring the features of the pen-shaped template includes:

[0105] Grayscale the Chinese character image and convert it into a grayscale image;

[0106] Normalize the grayscale image and adjust it to a uniform size of 32×32 pixels;

[0107] The gradient of the normalized 32×32 grayscale image is calculated, and the horizontal gradient G is calculated using the Sobel operator. x With the vertical gradient G y , specifically:

[0108] G x =I*S x ,G y =I*S y

[0109] Among them, I is the normalized grayscale image, S x 、S y They are horizontal and vertical Sobel filter kernels respectively, and * is the convolution operation;

[0110] Based on the horizontal gradient G x With the vertical gradient G y Calculate the gradient magnitude G(x,y) and gradient direction θ(x,y) of each pixel point. The expression is:

[0111]

[0112] Where G(x,y) is the gradient magnitude of the pixel point (x,y), and θ(x,y) is the gradient direction of the pixel point (x,y).

[0113] The image is divided into several 2×2 pixel cell units. For each pixel point in the cell, the gradient amplitude G(x,y) is projected to the corresponding interval of the predefined direction histogram according to its gradient direction, and the gradient direction histogram is statistically formed, as shown in the following example: Figure 6 As shown;

[0114] Multiple adjacent cell units are grouped into a block, and the gradient direction histogram of the block is normalized by L2-Hys to eliminate the influence of illumination and contrast. The normalization formula is:

[0115]

[0116] Among them, v normis the normalized gradient histogram vector, v is the gradient histogram vector within the block, and ∈ is a stable constant to prevent the denominator from being zero;

[0117] The normalized gradient direction histogram vectors of all blocks in the image are spliced in sequence to form the final pen-shaped template feature vector S h .

[0118] The dynamic time warping distance is:

[0119]

[0120] Among them, DTW(sp A ,sp B ) represents the dynamic time warping distance between the stroke sequence of the Chinese character image A to be recognized and the standard Chinese character image B, sp A (i) and sp B (π(i)) represents the i-th stroke vector of the Chinese character image A to be recognized and the corresponding π(i)-th stroke vector of the standard Chinese character image B on the alignment path π, where the stroke vector is obtained by extracting the start and end coordinates, direction, length and stroke morphological features of each stroke after segmenting the Chinese character image; ‖‖2 is the Euclidean norm, π is the alignment path, and the optimal alignment path is obtained by the dynamic time warping DTW algorithm; n = min(|sp A |,|sp B |) is the length of the shorter stroke sequence, |sp A | is the number of strokes of the Chinese character image A to be recognized, π(i) is the number of strokes sp in the path π A (i) Aligned sp B The stroke index in .

[0121] like Figure 4 As shown in Figure 2, the structural similarity calculation process is:

[0122] For the Chinese character images to be recognized and the standard Chinese character images, the character segmentation algorithm based on OpenCV is used to perform structural decomposition with the maximum decomposition degree, and each Chinese character is decomposed into several sub-components. Based on the decomposition results, a hierarchical structure tree is constructed. The structure number is as follows: Figure 7 As shown, each layer of the structure tree represents the decomposition level of Chinese characters, with the root node corresponding to the complete Chinese character and the leaf nodes corresponding to the basic components. In the structure tree, each node represents the sub-component features and their spatial positions at the corresponding level, and the sub-nodes of the node are the components that are further decomposed, forming a recursive tree structure.

[0123] Determine whether the structures of two Chinese characters are the same, provided that the topological structures of the structure trees are the same and the number and categories of subcomponents at the corresponding levels match; if this condition is not met, the structures are determined to be different;

[0124] When the structure is the same, calculate the weight attenuation coefficient ω of each level sub-component in the structure tree k The calculation formula is:

[0125]

[0126] Among them, ω k is the weight attenuation coefficient of the kth layer; K is the maximum decomposition level of the structure tree;

[0127] Based on the structure tree matching algorithm, the best matching mapping path σ of the structural component is obtained, which is used to map the kth subcomponent A of the Chinese character to be recognized. k Corresponding subcomponent B of standard Chinese characters σ(k) Perform alignment and calculate the structural alignment score based on the weight decay coefficient:

[0128]

[0129] Among them, align score is the structural alignment score, sim(A k ,B σ(k) ) represents the HOG feature similarity between the k-th sub-components;

[0130] Calculate the structural similarity score based on the structural alignment score:

[0131]

[0132] Among them, S struct is the structural similarity between the Chinese character image A to be recognized and the standard Chinese character image B, |A| and |B| are the total number of subcomponents of the Chinese character image A to be recognized and the standard Chinese character image B, b is the penalty coefficient for position difference, Δ pos To match the position differences of sub-components, the Euclidean distance of the corresponding sub-component center coordinates is calculated, specifically:

[0133]

[0134] in, and Subcomponent A k and B σ(k) The center coordinates of

[0135] When the structures are different, the Jaccard similarity of the radical set is calculated as the structural similarity:

[0136]

[0137] Among them, C A 、C Bare the radical sets of the Chinese character image A to be recognized and the standard Chinese character image B, respectively, where |C A ∩C B | represents the number of radicals shared by the two images, |C A ∪C B | is the union number of the radicals of the two images.

[0138] like Figure 5 As shown in Figure 2, the topological similarity calculation process includes:

[0139] Binarize and normalize the Chinese character image A to be recognized and the standard Chinese character image B to obtain binary images of uniform size;

[0140] The Zhang-Suen thinning algorithm is applied to binary images for skeleton extraction. The Zhang-Suen algorithm is based on pixel neighborhood iterative processing. Through a two-stage iterative rule, while maintaining image connectivity and structural integrity, it gradually removes edge pixels, ultimately retaining a single-pixel width skeleton image of the main stroke.

[0141] Perform pixel connectivity analysis on the generated skeleton graph, extract key pixels such as intersections and endpoints, and construct the topological graph structure G = (V, E) of the stroke skeleton, where: V = {v1, v2, ..., v m} is a node set, which represents the key points in the stroke skeleton graph, including intersections, endpoints, and turning points. Nodes are extracted from the skeleton graph through pixel connectivity detection; E = {e ij ∣(v i ,v j )∈V×V} is an edge set, which represents the connection relationship between nodes formed by stroke paths; each edge e ij Including the eigenvector f ij , including the length, direction, and curvature of the edge, which is used for subsequent weighted matching;

[0142] Constructing graph structure G A =(V A ,E A ),G B =(V B ,E B ), respectively representing the stroke skeleton diagrams of the Chinese character image A to be recognized and the standard Chinese character image B;

[0143] The maximum weight matching MWM algorithm is used to A =(V A ,E A ),G B =(V B ,E B ) for structural matching, for any pair of edges and The similarity weight is defined as:

[0144]

[0145] in, Represents the graph structure G respectively A =(V A ,E A ),G B =(V B ,E B ) in the two stroke path edges, represents the Euclidean distance between the eigenvectors of two edges, and β is a positive control coefficient;

[0146] Construct a complete bipartite graph G M =(E A ,E B ,W), perform the maximum weight matching MWM algorithm on it to obtain the optimal matching set, and calculate the maximum value of the total weight in the complete bipartite graph matching:

[0147]

[0148] Among them, M is the edge matching set, MWM(G A ,G B ) is the graph structure G A =(V A ,E A ),G B =(V B ,E B ) The total similarity score under maximum matching;

[0149] Calculate the topological similarity based on the total similarity score under the maximum matching:

[0150]

[0151] Among them, ‖E A ‖,‖E B ‖ respectively represent the graph structure G A =(V A ,E A ), G B =(V B ,E B ) is the number of edges.

[0152] Step S5: Calculating the Chinese character similarity score between the Chinese character image to be recognized and the standard Chinese character image based on the fusion of stroke similarity, structural similarity and topological similarity;

[0153] Step S6: constructing a joint loss function including classification loss and similarity loss based on the Chinese character category prediction result and the Chinese character similarity score;

[0154] The Chinese character similarity score formula is:

[0155] sim(A, B)=c·S stroje +d·S struct +(1-cd)·S topo

[0156] Among them, sim(A, B) is the Chinese character similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, S stroke 、S struct 、S topo are the stroke similarity, structural similarity and topological similarity between the Chinese character image A to be recognized and the standard Chinese character image B, respectively. c and d are weight coefficients.

[0157] Step S7: Perform back propagation training on the Chinese character classifier network based on the joint loss function to obtain the trained Chinese character classifier network to realize the recognition of Chinese characters.

[0158] The joint loss function is:

[0159]

[0160] in, is the joint loss function, C is the total number of Chinese character categories, y i Represents the true category label of the i-th Chinese character category. When the Chinese character image to be recognized is the i-th Chinese character category, y i =1, the rest are 0, is the probability of the i-th category predicted by the classifier, sim(A, B) is the Chinese character similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, and λ1 and λ2 are weight coefficients.

[0161] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A Chinese character recognition method based on joint optimization of Chinese character similarity algorithm, characterized in that: The following steps are involved: Obtain the Chinese character image to be recognized and perform preprocessing; Inputting the preprocessed image into a Chinese character classifier network and outputting a corresponding Chinese character category prediction result; Obtaining a standard Chinese character image corresponding to the Chinese character category prediction result from the constructed Chinese character image database; respectively calculating the stroke similarity, structural similarity and topological similarity between the Chinese character image to be recognized and the standard Chinese character image; Calculating a Chinese character similarity score between the Chinese character image to be recognized and the standard Chinese character image based on the fusion of the stroke similarity, structural similarity and topological similarity; A joint loss function including classification loss and similarity loss is constructed based on the Chinese character category prediction results and the Chinese character similarity scores; The Chinese character classifier network is back-propagated and trained based on the joint loss function to obtain the trained Chinese character classifier network, thereby realizing the recognition of Chinese characters.

2. A Chinese character recognition method based on joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The preprocessing includes binarization, normalization and grayscale processing of the image to obtain a standard image input of uniform size.

3. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The Chinese character classifier network is an image classification model based on a convolutional neural network (CNN), which specifically includes multiple convolutional layers, pooling layers, normalization layers, and fully connected layers.

4. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The stroke similarity calculation formula is: Among them, S stroke is the stroke similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, a is the weight coefficient, γ is a positive real number adjustment parameter, Sh A 、Sh B are the pen shape template features of the Chinese character image A to be recognized and the standard Chinese character image B, SSIM (Sh A ,Sh B ) is the cosine similarity of the pen shape template features of the Chinese character image A to be recognized and the standard Chinese character image B, DTW(sp A ,sp B ) is the dynamic time warping distance, which is used to measure the similarity between the sequence sequences of two images.

5. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 4 is characterized in that: The pen-shaped template feature acquisition process includes: Grayscale the Chinese character image and convert it into a grayscale image; Normalize the grayscale image and adjust it to a uniform size of 32×32 pixels; The gradient of the normalized 32×32 grayscale image is calculated, and the horizontal gradient G is calculated using the Sobel operator. x With the vertical gradient G y , specifically: G x =I*S x ,G y =I*S y Among them, I is the normalized grayscale image, S x 、S y They are horizontal and vertical Sobel filter kernels respectively, and * is the convolution operation; Based on the horizontal gradient G x With the vertical gradient G y Calculate the gradient magnitude G(x,y) and gradient direction θ(x,y) of each pixel point. The expression is: Where G(x,y) is the gradient magnitude of the pixel point (x,y), and θ(x,y) is the gradient direction of the pixel point (x,y). The image is divided into several 2×2 pixel cell units. For each pixel point in the cell, the gradient amplitude G(x,y) is projected to the corresponding interval of the predefined direction histogram according to its gradient direction, and the gradient direction histogram is statistically formed. Multiple adjacent cell units are grouped into a block, and the gradient direction histogram of the block is normalized by L2-Hys to eliminate the influence of illumination and contrast. The normalization formula is: Among them, v norm is the normalized gradient histogram vector, v is the gradient histogram vector within the block, and ∈ is a stable constant to prevent the denominator from being zero; The normalized gradient direction histogram vectors of all blocks in the image are spliced in sequence to form the final pen-shaped template feature vector S h .

6. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 4 is characterized in that: The dynamic time warping distance is: Among them, DTW(sp A ,sp B ) represents the dynamic time warping distance between the stroke sequence of the Chinese character image A to be recognized and the standard Chinese character image B, sp A (i) and sp B (π(i)) represents the i-th stroke vector of the Chinese character image A to be recognized and the corresponding π(i)-th stroke vector of the standard Chinese character image B on the alignment path π, where the stroke vector is obtained by extracting the start and end coordinates, direction, length and stroke morphological features of each stroke after segmenting the Chinese character image; ‖‖2 is the Euclidean norm, π is the alignment path, and the optimal alignment path is obtained by the dynamic time warping DTW algorithm; n = min(|sp A |,|sp B |) is the length of the shorter stroke sequence, |sp A | is the number of strokes of the Chinese character image A to be recognized, π(i) is the number of strokes sp in the path π A (i) Aligned sp B The stroke index in .

7. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The structural similarity calculation process is: For the Chinese character images to be recognized and the standard Chinese character images, a character segmentation algorithm based on OpenCV is used to perform structural decomposition with the maximum decomposition degree. Each Chinese character is decomposed into several sub-components. Based on the decomposition results, a hierarchical structure tree is constructed. Each layer of the structure tree represents the decomposition level of the Chinese character. The root node corresponds to the complete Chinese character, and the leaf node corresponds to the basic component. In the structure tree, each node represents the sub-component features and spatial position of the corresponding level. The child nodes of the node are the further decomposed components, forming a recursive tree structure. Determine whether the structures of two Chinese characters are the same, provided that the topological structures of the structure trees are the same and the number and categories of subcomponents at the corresponding levels match; if this condition is not met, the structures are determined to be different; When the structure is the same, calculate the weight attenuation coefficient ω of each level sub-component in the structure tree k The calculation formula is: Among them, ω k is the weight attenuation coefficient of the kth layer; K is the maximum decomposition level of the structure tree; Based on the structure tree matching algorithm, the best matching mapping path σ of the structural component is obtained, which is used to map the kth subcomponent A of the Chinese character to be recognized. k Corresponding subcomponent B of standard Chinese characters σ(k) Perform alignment and calculate the structural alignment score based on the weight decay coefficient: Among them, align score is the structural alignment score, sim(A k ,B σ(k) ) represents the HOG feature similarity between the k-th sub-components; Calculate the structural similarity score based on the structural alignment score: Among them, S struct is the structural similarity between the Chinese character image A to be recognized and the standard Chinese character image B, |A| and |B| are the total number of subcomponents of the Chinese character image A to be recognized and the standard Chinese character image B, b is the penalty coefficient for position difference, Δ pos To match the position differences of sub-components, the Euclidean distance of the corresponding sub-component center coordinates is calculated, specifically: in, and Subcomponent A k and B σ(k) The center coordinates of When the structures are different, the Jaccard similarity of the radical set is calculated as the structural similarity: Among them, C a 、C B are the radical sets of the Chinese character image A to be recognized and the standard Chinese character image B, respectively, where |C A ∩C B | represents the number of radicals shared by the two images, |C A ∪C B | is the union number of the radicals of the two images.

8. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The topological similarity calculation process includes: Binarize and normalize the Chinese character image A to be recognized and the standard Chinese character image B to obtain binary images of uniform size; The Zhang-Suen thinning algorithm is applied to the binary image for skeleton extraction. The Zhang-Suen algorithm is based on pixel neighborhood iterative processing. Through a two-stage iterative rule, edge pixels are gradually removed while maintaining image connectivity and structural integrity, ultimately retaining a single-pixel width skeleton image of the stroke trunk; Perform pixel connectivity analysis on the generated skeleton graph, extract key pixels such as intersections and endpoints, and construct the topological graph structure G = (V, E) of the stroke skeleton, where: V = {v1, v2, ..., v m } is a node set, representing the key points in the stroke skeleton graph, including intersections, endpoints, and turning points. The nodes are extracted from the skeleton graph through pixel connectivity detection; E = {e ij ∣(v i ,v j )∈V×V} is an edge set, which represents the connection relationship between nodes formed by stroke paths; each edge e ij Including the eigenvector f ij , including the length, direction, and curvature of the edge, which is used for subsequent weighted matching; Constructing graph structure G A =(V A ,E A ),G B =(V B ,E B ), respectively representing the stroke skeleton diagrams of the Chinese character image A to be recognized and the standard Chinese character image B; The maximum weight matching MWM algorithm is used to A =(V A ,E A ),G B =(V B ,E B ) for structural matching, for any pair of edges and The similarity weight is defined as: in, Represents the graph structure G respectively A =(V A ,E A ),G B =(V B ,E B ) in the two stroke path edges, represents the Euclidean distance between the eigenvectors of two edges, and β is a positive control coefficient; Construct a complete bipartite graph G M =(E A ,E B ,W), perform the maximum weight matching MWM algorithm on it to obtain the optimal matching set, and calculate the maximum value of the total weight in the complete bipartite graph matching: Among them, M is the edge matching set, MWM(G A ,G B ) is the graph structure G A =(V A ,E A ),G B =(V B ,E B ) The total similarity score under maximum matching; Calculate the topological similarity based on the total similarity score under the maximum matching: Among them, ‖E A ‖,‖E B ‖ respectively represent the graph structure G A =(V A ,E A ), G B =(V B ,E B ) is the number of edges.

9. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The Chinese character similarity score formula is: sim(A,B)=c·S stroje +d·S struct +(1-c-d)·S topo Among them, sim(A, B) is the Chinese character similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, S stroke 、S struct 、S topo are the stroke similarity, structural similarity and topological similarity between the Chinese character image A to be recognized and the standard Chinese character image B, respectively. c and d are weight coefficients.

10. The Chinese character recognition method based on the joint optimization of Chinese character similarity algorithm according to claim 1, characterized in that: The joint loss function is: in, is the joint loss function, C is the total number of Chinese character categories, y i Represents the true category label of the i-th Chinese character category. When the Chinese character image to be recognized is the i-th Chinese character category, y i =1, the rest are 0, is the probability of the i-th category predicted by the classifier, sim(A, B) is the Chinese character similarity score between the Chinese character image A to be recognized and the standard Chinese character image B, and λ1 and λ2 are weight coefficients.

Images