A zero-shot seal character recognition method based on Chinese character root structure

Abstract

The present invention discloses a zero-sample Xiaozhuan recognition method based on Chinese character root structure. The present invention first selects Xiaozhuan and traditional Chinese character set pictures with matching text labels as training sets and test sets; then constructs a Xiaozhuan recognition model, which includes an encoder and two parallel sub-network branches; the encoder parses the Xiaozhuan picture into a high-dimensional feature vector; the two parallel sub-network branches include a "Xiaozhuan-Traditional Conversion Generation Network" branch and an "Attention-based Chinese Character Root Structure Recognition Network" branch; branch one includes two decoders, which obtain corresponding traditional Chinese character pictures and Xiaozhuan pictures according to the high-dimensional feature vector; branch two obtains the Chinese character root structure vector of Xiaozhuan according to the high-dimensional feature vector, thereby recognizing Xiaozhuan. The present invention converts the "Xiaozhuan recognition problem" into a "Traditional Chinese character recognition problem" with a more regular root structure, reduces the difficulty of Xiaozhuan recognition, and provides a new idea for the zero-sample Xiaozhuan recognition task.

Description

Technical Field

[0001] The present invention relates to the field of image and text recognition and deep learning technology, and in particular to a zero-sample small seal character recognition method based on the root structure of Chinese characters. Background Art

[0002] Traditional Chinese character recognition technology is a branch of pattern recognition. The basic process is to pre-process the input text image, extract its features, compare and distinguish its features with each feature template stored in the recognition dictionary, calculate the similarity of the matching text, and finally take the standard text with the greatest similarity as the recognition result. With the support of a large number of annotated data sets, the existing Chinese character recognition technology has a high recognition accuracy rate in the fields of simplified and traditional Chinese printed characters.

[0003] Xiaozhuan is a unified Chinese writing form created by Qin Shihuang after he unified the six kingdoms. It plays an important role in the history of Chinese writing development. Xiaozhuan often appears in the trademarks and slogans of many companies. However, compared with the more mature printed simplified and traditional Chinese character recognition technologies with high recognition accuracy, there are currently fewer image and text recognition technologies for Xiaozhuan. This is mainly due to the following two difficulties in Xiaozhuan recognition: 1. Compared with simplified and traditional Chinese characters, the number of data sets of Xiaozhuan character libraries is relatively small, and it is currently difficult to find root character libraries for Xiaozhuan on the market, which makes it very difficult to match the features of image and text recognition; 2. Compared with simplified and traditional Chinese characters, Xiaozhuan is more pictographic and more symbolic, so its structure is more complex, has more redundant information, and it is more difficult to extract structural information. Therefore, the above-mentioned traditional Xiaozhuan image and text recognition technology, which directly matches the image feature information with the Xiaozhuan standard character library for similarity, has a low recognition accuracy for Xiaozhuan and often cannot correctly recognize the simplified Chinese characters corresponding to Xiaozhuan. Summary of the invention

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a zero-sample small seal character recognition method based on the root structure of Chinese characters.

[0005] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0006] First, select the pictures of small seal characters and traditional characters with matching text labels as training sets and test sets; then construct a small seal character recognition model, which includes an encoder and two parallel sub-network branches; the encoder parses the small seal character picture into an encoder of a high-dimensional feature vector. The two parallel sub-network branches include a "small seal character-traditional conversion generation network" branch and an "attention-based Chinese character root structure recognition network" branch; branch one includes two decoders, which obtain the corresponding traditional Chinese character pictures and small seal characters according to the high-dimensional feature vector to assist in the generation of the high-dimensional feature vector of the small seal character; branch two obtains the Chinese character root structure vector of the small seal character according to the high-dimensional feature vector, thereby recognizing the small seal character; finally, set the model loss function so that the two branches of the model assist and restrict each other, thereby obtaining a higher accuracy rate of small seal character recognition. The present invention transforms the "small seal character recognition problem" into a "traditional character recognition problem" with a more regular root structure, greatly reducing the difficulty of small seal character recognition and providing a new idea for the zero-sample small seal character recognition task.

[0007] Furthermore, the construction of the training set and the test set is as follows:

[0008] 1-1. Select N small seal characters T with similar structural contents to their corresponding traditional characters from the small seal character library, sort the small seal character set T according to the shape structure of each small seal character and the number of occurrences of its radicals, select N1 small seal characters T1 so that the number of occurrences of each radical in T1 is as uniform as possible, and remove the remaining N2 characters in T from the T1 character set as the character set T2;

[0009] 1-2. Generate N1 small seal characters images with font A1 and content from character set T1, where A1 is usually "Founder Small Seal Characters" font; generate corresponding traditional Chinese characters images with font B and content from character set T1, where B is usually "Founder Handwriting - European Traditional Characters" font, and splice the N1 small seal characters images with font A1 and the corresponding traditional Chinese characters images with font B from character set T1 as training set TR;

[0010] 1-3. Build test sets TE1, TE2, TE3:

[0011] N2 pictures of small seal characters in A1 font and pictures of traditional Chinese characters in B font of the character set T2 are correspondingly spliced ​​as training set TE1; N21 pictures of small seal characters in A2 font and pictures of traditional Chinese characters in B font of the character set T1 are correspondingly spliced ​​as training set TE2, wherein A2 is usually the "Han Yi Zhuan Shu Fan" font; N2 pictures of small seal characters in A2 font and pictures of traditional Chinese characters in B font of the character set T2 are correspondingly spliced ​​as training set TE3;

[0012] 1-4. Collect the shape structure and radical structure of the traditional Chinese characters contained in the character set T, and encode all the shape structures and Chinese character radicals, each shape structure and each Chinese character radical has a unique code, find the shape structure and radical corresponding to the traditional Chinese characters in the character set T, convert each shape structure and radical into its corresponding coding sequence, add the numbers 2 and 3 at the beginning and end of the code as the label start and end marks of the text recognition network, and finally use 0 at the end of the sequence to fill the coding sequence to a length of 17 labels as the predicted label for each data set image.

[0013] Furthermore, the encoder E in the small seal character recognition model is constructed as follows:

[0014] The encoder E is composed of eight convolution modules e1~e8, the convolution kernel size of the e1~e8 convolution modules is set to 2×2, the step size is set to 2, and the padding is set to SAME; according to the direction of forward propagation, the number of convolution kernels of the e1~e8 convolution modules is set to 64, 128, 256, 512, 512, 512, 512 and 512 respectively; and a layer of LReLu mapping is added before each convolution module of e2~e8 to complete the nonlinear transformation of the data, and a batch normalization layer BN is added after each convolution module of e2~e8; the feature array after processing by the e8 layer is called data1, and the feature array after processing by the e6 layer is called data2.

[0015] Furthermore, the construction of the Xiaozhuan-Traditional Chinese conversion generation network branch in the Xiaozhuan recognition model is as follows:

[0016] The Xiaozhuan-Traditional conversion generation network branch consists of two decoders G1 and G2 with completely identical structures and two corresponding discriminators D1 and D2 with completely identical structures; G1 is used to decode the high-dimensional feature vector generated by the encoder E to generate its corresponding Traditional Chinese image, and D1 is used to discriminate between the input Traditional Chinese image and the generated Traditional Chinese image as a real image or a false image, and feed the discrimination result back to the encoder E, thereby improving the image decoding quality of the encoder E and the Traditional Chinese image generation quality of G1; G2 and D2 have similar functions to G1 and D1, and are respectively used to generate Xiaozhuan images according to the high-dimensional feature vector and discriminate the quality of Xiaozhuan images.

[0017] Furthermore, the decoder includes eight deconvolution modules g1~g8, the convolution kernel size of the g1~g8 convolution modules is set to 2×2, the step size is set to 2, and the number of convolution kernels of the g1~g8 convolution modules is set to 512, 512, 512, 512, 512, 256, 128 and 64 respectively according to the direction of forward propagation; a layer of ReLU mapping is added before each deconvolution module of g1~g8 to complete the nonlinear transformation of the data, and a BN layer is added after each deconvolution module of g1~g7. After BN, each deconvolution module of g1~g7 is connected with the e7~e1 modules in the encoder E respectively, g1~g3 needs to perform a dropout operation before the concat operation, and the probability is set to 0.5. Finally, a tanh activation operation is performed on the g8 deconvolution module structure, so that the model converges quickly.

[0018] Furthermore, the discriminator includes 4 convolution modules d1~d4, each of which adopts the structure in CNN, the convolution kernel size of the d1~d4 convolution modules is set to 2×2, the step size is set to 2, and the padding is set to SAME; according to the direction of forward propagation, the number of convolution kernels of the d1~d4 convolution modules is set to 64, 128, 256 and 512 respectively; a layer of LReLU mapping is added after each convolution module of d1~d4, and a BN layer is added after the LReLU mapping of the d2~d4 convolution modules, and a fully connected layer is added after the d4 convolution module to judge whether the image is true or false.

[0019] Furthermore, the attention-based Chinese character root structure recognition network branch adopts the Transformer decoding layer structure; the data2 feature data with dimensions of 16, 4, 4 and 512 generated by the encoder E is reshaped into 16, 16 and 512 dimensions as network input; the step size of the Transformer decoding layer is set to 16, the dimension is 512, and the prediction categories are 409 in total, of which 4 are mask marks and 405 are the number of font root categories; finally, the probabilities of 16 Chinese character root category predictions are output, and the position with the largest probability is taken as the root prediction result of the step size, and finally a feature vector representing the semantic information of the Chinese character root structure is obtained; the above-mentioned Chinese character root category prediction results are compared with the IDS dictionary using the K nearest neighbor method to obtain the final prediction result.

[0020] Furthermore, the loss function of the small seal character recognition model is designed as follows:

[0021] The Xiaozhuan-Traditional conversion generation network branch mainly includes two parts: the discriminator loss and the generator loss. The discriminator loss is used to distinguish the authenticity of the generated image and the original image, wherein the D1 discriminator is used to distinguish the authenticity of the Traditional Chinese image, and the D2 is used to distinguish the authenticity of the Xiaozhuan image.

[0022] The G1 and G2 losses consist of three parts: cycle consistency loss, feature matching loss, and generative adversarial network loss; the calculation formula of the target loss function combining the three is:

[0023] L(X1,G(x),D(x),x,y)=αL p (G(x),y)+βL c (E(x))+L G (D(G(x)))

[0024] Among them, x is the input image to be identified for generation, y is the target image, X1 is the high-dimensional feature vector, G(x) is the generated image, and D(x) is the discrimination result; α and β are L p , L c The weight value, L p To determine the cycle consistency loss, make the generated image as consistent as possible with the target image, L c For feature loss matching, the high-dimensional features of small seal characters and traditional characters are consistent. G To generate the loss function of the adversarial network, the least squares loss is used;

[0025] In order to make full use of the semantic information of the similar structure between Xiaozhuan and traditional Chinese characters, the feature vectors of data1 obtained by encoder E encoding Xiaozhuan and traditional Chinese images should be as consistent as possible; at the same time, data2 is used as the input of the attention-based Chinese character root structure recognition network branch, and its output feature vectors should be as consistent as possible. c The description is as follows:

[0026] L c =‖Data2(x1)-Data2(x2)‖1+‖Data1(x1)-Data1(x2)‖1

[0027] Wherein, Data2 represents the data2, Data1 represents the data1, x1 and x2 represent the input seal characters and traditional characters images respectively;

[0028] The attention-based Chinese character root structure recognition network branch uses the normalized exponential function cross entropy as the loss function to calculate the consistency loss between the output semantic feature vector and the traditional Chinese character root structure vector.

[0029] The beneficial effects of the present invention are as follows:

[0030] The present invention provides a new idea for the recognition technology of small seal script. As described in the background technology, the structure of small seal script is complex and has more redundant information, so the recognition rate of traditional small seal script image and text recognition technology is relatively low. Traditional Chinese characters, as a Chinese writing system generated after the evolution of small seal script into official script, have great similarity with small seal script in structure. Therefore, after the small seal script picture to be recognized is sent into the encoder, its corresponding traditional Chinese character picture is sent into the same encoder, and the feature vector obtained by encoding the small seal script picture is controlled to be consistent with the feature vector obtained by encoding the traditional Chinese character picture, so that the structural correspondence between small seal script and traditional Chinese characters can be used to remove the redundant information such as strokes and pictographic meanings in small seal script. And the task of "small seal script image and text recognition" is cleverly converted into the task of "traditional character image and text recognition", thereby making full use of the rich root data set resources of traditional Chinese characters with more regular root structures, and only using less small seal script sample data can complete the zero-sample small seal script recognition task with a high recognition accuracy.

[0031] In order to efficiently complete the above encoding task, the present invention proposes a novel picture generation model, which can make full use of the semantic information of the similar structure between small seal characters and traditional characters. The small seal character picture and its corresponding traditional character picture are respectively input into the model. The model has an encoder E, which can ignore the redundant information of the input font, focus on its structural information, and encode the input picture into a high-dimensional feature vector; the model has two decoders, one decoder G1 specifically responsible for decoding and generating small seal characters, and the other decoder G2 specifically responsible for decoding and generating traditional characters. The high-dimensional feature vector containing the picture structure information encoded by the encoder is input into G1 and G2 respectively, and high-quality small seal characters and traditional characters can be obtained respectively. The generation effects of the pictures are mutually constrained, so as to obtain high-quality generated pictures, and at the same time, the encoding ability of E is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the structure of a zero-sample seal character recognition model based on the root structure of Chinese characters;

[0033] Figure 2 It is a schematic diagram of the main steps of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The embodiment of the present invention is based on deep learning related technologies and realizes a method for zero-sample small seal recognition based on the root structure of Chinese characters. This network includes an encoder E that parses small seal images into high-dimensional feature vectors, and two parallel sub-network branches. The first branch can obtain the corresponding traditional Chinese characters and small seal images according to the above high-dimensional feature vectors to assist in the generation of high-dimensional feature vectors of small seal. The second branch can obtain the Chinese character root structure vector of small seal according to the above high-dimensional feature vectors, thereby identifying small seal. The present invention not only improves the recognition efficiency of small seal, but also improves the problem of severe shortage and uneven distribution of small seal data sets, and provides a new idea for zero-sample recognition of small seal. Figure 1 and 2 As shown, the specific scheme of the seal character recognition method provided by the present invention is as follows:

[0036] Step S1: construct training set and test set, i.e. data construction module;

[0037] Step S2: constructing a small seal character recognition model, i.e., a small seal character recognition model construction module;

[0038] Step S3: designing the loss function of the small seal character recognition model, i.e., the loss function construction module;

[0039] Step S4: Use the training set to train the Xiaozhuan recognition model. When the loss function value is less than the threshold, the final Xiaozhuan recognition model is obtained; that is, the model training module;

[0040] Step S5: input the seal script images in the test set into the final seal script recognition model to obtain the traditional Chinese character generation images corresponding to the seal script images and the seal script recognition results; i.e., the model recognition module;

[0041] The step S1 constructs a training set and a test set, and the specific method is as follows:

[0042] 1-1. Select 5475 small seal characters T with similar structural contents to their corresponding traditional characters from the small seal character library, sort the small seal character set T according to the shape structure of each small seal character and the number of occurrences of its radicals, select 4380 small seal characters T1 so that the number of occurrences of each radical in T1 is as uniform as possible, and remove the remaining 1095 characters in T from the T1 character set as the character set T2;

[0043] 1-2. Generate 4380 small seal characters images with font A1 and content from character set T1, where A1 is usually "Founder Small Seal Character" font; generate corresponding traditional Chinese characters images with font B and content from character set T1, where B is usually "Founder Handwriting - European Traditional Character" font, and splice the 4380 small seal characters images with font A1 and the corresponding traditional Chinese characters images with font B from character set T1 as training set TR;

[0044] 1-3. Using the same method as above to construct the training set TR, construct the test sets TE1, TE2, and TE3 respectively:

[0045] 1095 A1 font small seal characters images and B font traditional Chinese characters images of the character set T2 are correspondingly spliced ​​as training set TE1; 4380 A2 font small seal characters images and B font traditional Chinese characters images of the character set T1 are correspondingly spliced ​​as training set TE2, wherein A2 is usually the "Han Yi Zhuan Shu Fan" font; 1095 A2 font small seal characters images and B font traditional Chinese characters images of the character set T2 are correspondingly spliced ​​as training set TE3;

[0046] 1-4. Collect the shape structure and radical structure of the traditional Chinese characters contained in the character set T, and encode all the shape structures and Chinese character radicals, each shape structure and each Chinese character radical has a unique code, use the integrated dictionary system dictionary ("IDS dictionary") in Zhang, J., J. Du, and L. Dai. "Radical Analysis Network for Learning Hierarchies of Chinese Characters." to find the shape structure and radical corresponding to the traditional Chinese characters in the character set T, convert each shape structure and radical into its corresponding encoding sequence, add the numbers 2 and 3 at the front and back of the code as the label start and end marks of the text recognition network, and finally use 0 at the end of the sequence to fill the encoding sequence to a length of 17 labels as the predicted label of each data set image.

[0047] The step S2 constructs a small seal recognition model, and the specific method is as follows:

[0048] 2-1. First, build an encoder E that can ignore redundant information such as strokes of the seal script image and efficiently extract its structural information:

[0049] The encoder E is composed of eight convolution modules e1 to e8, each of which adopts the structure of convolutional neural network (CNN) as an existing mature technology. The convolution kernel size of the e1 to e8 convolution modules is set to 2×2, the step size is set to 2, and the padding is set to SAME. According to the direction of forward propagation, the number of convolution kernels of the e1 to e8 convolution modules is set to 64, 128, 256, 512, 512, 512, 512 and 512 respectively. In order to maintain the stability of data distribution and avoid problems such as gradient explosion and gradient disappearance, a layer of Leaky ReLu (LReLu) mapping is added before each convolution module of e2 to e8 to complete the nonlinear transformation of data, and a batch normalization layer (BN) is added after each convolution module of e2 to e8, wherein the LReLu and BN layers are existing mature technologies. In particular, for the convenience of subsequent description, the feature array processed by the e8 layer is called data1, and the feature array processed by the e6 layer is called data2.

[0050] 2-2. Then, in order to efficiently complete the above encoding task and make full use of the semantic information of the structural similarity between small seal characters and traditional characters, the present invention constructs two branches of the "small seal character-traditional character conversion generation network" model and the "attention-based Chinese character root structure recognition network" model that are carried out in parallel.

[0051] The Xiaozhuan-Traditional conversion generation network model consists of two decoders G1 and G2 with completely identical structures and two corresponding discriminators D1 and D2 with completely identical structures. G1 is used to decode and generate the corresponding traditional Chinese image according to the high-dimensional feature data generated by E, and D1 is used to distinguish the input traditional Chinese image and the generated traditional Chinese image as a real image or a false image, and feed the discrimination result back to E, so as to improve the image decoding quality of E and the traditional Chinese image generation quality of G1; G2 and D2 have similar functions to G1 and D1, and are used to generate Xiaozhuan images according to the high-dimensional feature data and to judge the quality of Xiaozhuan images respectively.

[0052] The decoder includes eight deconvolution modules g1 to g8, where deconvolution is an existing mature technology. The convolution kernel size of the g1 to g8 convolution modules is set to 2×2, the step size is set to 2, and the padding is set to the same size as the picture. According to the direction of forward propagation, the number of convolution kernels of the g1 to g8 convolution modules is set to 512, 512, 512, 512, 512, 256, 128 and 64 respectively. In order to maintain the stability of data distribution and avoid problems such as gradient explosion and gradient disappearance, a layer of ReLU mapping is added before each deconvolution module of g1 to g8 to complete the nonlinear transformation of the data, and a BN layer is added after each deconvolution module of g1 to g7, where ReLU is an existing mature technology. In order to generate high-quality images, after BN, each deconvolution module of g1 to g7 is connected (concat) with the e7 to e1 modules in E. In particular, to prevent overfitting, g1 to g3 need to perform a dropout operation before the concat operation, with a probability set to 0.5. The concat operation and the dropout operation are existing mature technologies. Finally, the tanh activation operation is performed on the g8 deconvolution module structure to make the model converge quickly.

[0053] The discriminator includes 4 convolution modules d1~d4, each of which adopts the structure of CNN. The convolution kernel size of the convolution modules d1~d4 is set to 2×2, the step size is set to 2, and the padding is set to SAME. According to the direction of forward propagation, the number of convolution kernels of the convolution modules d1~d4 is set to 64, 128, 256 and 512 respectively. Similar to the E structure, a layer of LReLU mapping is added after each convolution module d1~d4, and a BN layer is added after the LReLU mapping of the convolution modules d2~d4, and a fully connected layer is added after the d4 convolution module to judge the authenticity of the picture, wherein the fully connected layer is an existing mature technology.

[0054] The attention-based Chinese character root structure recognition network branch adopts the decoding layer structure of the Transformer in Vaswani, Ashish, et al. Attention Is All You Need., which is an existing mature technology. The data2 feature data with dimensions of 16, 4, 4 and 512 generated by the encoder E is reshaped into 16, 16 and 512 dimensions as network input. The step size of the Transformer decoding layer is set to 16, the dimension is 512, and there are 409 prediction categories in total, of which 4 are mask marks and 405 are the number of font root categories. Finally, the probabilities of the 16 Chinese character root category predictions are output, and the position with the largest probability is taken as the root prediction result of the step size, and finally a feature vector representing the semantic information of the Chinese character root structure is obtained. Then, the K nearest neighbor method (KNN) is used to compare the above Chinese character root category prediction results with the IDS dictionary to obtain the final prediction result, where KNN is an existing mature technology, such as Figure 1 shown.

[0055] The step S3 defines the loss function of the small seal recognition model, and the specific method is as follows:

[0056] In the embodiment of the present invention, the two parallel branches have their own loss functions to help the network converge quickly and improve the network training efficiency;

[0057] Firstly, the main purpose of setting the loss function for the branch of the small seal script-traditional Chinese conversion generation network is to utilize the structural similarity between small seal script and traditional Chinese characters. According to the feature data output by E, the small seal script image and its corresponding traditional Chinese character image are decoded and generated respectively, so that the small seal script-traditional Chinese characters can be migrated to each other. The more realistic the traditional Chinese character image generated by the branch is, the more accurate the feature extraction effect of E is, and therefore the recognition accuracy of the other "attention-based Chinese character root structure recognition network" branch is higher.

[0058] The branch mainly includes two parts: discriminator loss and generator loss:

[0059] The discriminator loss is used to distinguish the authenticity of the generated image and the original image, wherein the D1 discriminator is used to distinguish the authenticity of the traditional Chinese image, and the D2 is used to distinguish the authenticity of the seal script image;

[0060] The G1 and G2 losses consist of three parts: cycle consistency loss, feature matching loss, and generative adversarial network loss; the calculation formula of the target loss function combining the three is:

[0061] L(X1,G(x),D(x),x,y)=αL p (G(x),y)+βL c (E(x))+LG (D(G(x)))

[0062] Among them, x is the input image to be identified for generation, y is the target image, X1 is the high-dimensional feature vector, G(x) is the generated image, and D(x) is the discrimination result. α and β are L p , L c The weight value, L p To determine the cycle consistency loss, make the generated image as consistent as possible with the target image, L c For feature loss matching, the high-dimensional features of small seal characters and traditional characters are consistent. G is the loss function of the generative adversarial network. Preferably, in the embodiment of the present invention, α=100, β=15, L G The least squares loss is adopted.

[0063] In order to make full use of the semantic information of the similar structure between Xiaozhuan and traditional Chinese characters, the feature vectors of data1 obtained by E-encoding Xiaozhuan and traditional Chinese images should be kept as consistent as possible. At the same time, the feature vectors of data2 as the input of the attention-based Chinese character root structure recognition network model should also be as consistent as possible. The description of Lc is as follows:

[0064] L c =‖Data2(x1)-Data2(x2)‖1+‖Data1(x1)-Data1(x2)‖1

[0065] Among them, Data2 represents the data2, Data1 represents the data1, x1 and x2 represent the input small seal and traditional Chinese images respectively.

[0066] Secondly, the main purpose of setting the loss function of the attention-based Chinese character root structure recognition network branch is to help the model recognize the high-dimensional feature information of small seal script. The lower the loss function value, the more accurate the Chinese character root structure vector generated by the recognition, the higher the image generation quality of the small seal script-traditional Chinese character conversion generation network branch, and the better the mutual migration effect between small seal script and traditional Chinese character. The normalized exponential function (softmax) cross entropy is selected as the loss function to calculate the consistency loss between the output semantic feature vector and the traditional Chinese character root structure vector.

[0067] The specific method of step 4 for training the seal character recognition network is as follows:

[0068] In order to verify the feasibility of zero-sample Xiaozhuan recognition, the 4380 Xiaozhuan characters of TR and their corresponding traditional Chinese data sets were used to train the Xiaozhuan recognition model. In particular, for each epoch, the order of the data sets was randomly shuffled, and the data sets were enhanced in terms of position, size, angle, etc. When the loss function was less than the set threshold, the training was stopped and the final Xiaozhuan recognition model was obtained.

[0069] The specific method of testing the Xiaozhuan recognition network in step 5 is as follows:

[0070] The test sets TE1, TE2 and TE3 are sequentially placed into the trained seal script recognition model to obtain recognition results. The accuracy of the test set TE2 of the present invention is as high as 90.78%, and it can recognize seal script samples of different fonts with learned radical structures with high precision. In the case of zero samples, the recognition accuracy of the test sets TE1 and TE3 are 69.32% and 67.49%, as shown in Table 1.

[0071] Table 1 shows the comparison results of recognition accuracy between the network without G2 and RTN:

[0072]

[0073] The above RTN network is taken from "A Transformer-based Radical Analysis Network for Chinese Character Recognition", which is a network that can recognize Chinese character radicals. Compared with the present invention, it does not have a branch of a picture generation network of small seal characters and their corresponding traditional characters, that is, it does not use the meaning information of the structural correspondence between small seal characters and traditional characters. Therefore, the recognition accuracy obtained by using the RTN network for the task of recognizing small seal characters is very low. At the same time, the network after removing G2, that is, the recognition rate of small seal characters in the adversarial generation network with only a single decoder is greatly reduced. It can be seen that using two decoders G1 and G2 to restrict and strengthen E from the structural correspondence between small seal characters and traditional characters can greatly improve the recognition rate of small seal characters. Therefore, the two branches of the small seal character recognition model in the present invention are indispensable, and the structure of one decoder corresponding to two encoders can make full use of the meaning information of the structural correspondence between small seal characters and traditional characters, greatly improving the recognition accuracy of small seal characters.

Claims

1. A zero-sample seal character recognition method based on Chinese character root structure, characterized in that Firstly, select the pictures of small seal characters and traditional characters with matching text labels as training sets and test sets; then construct a small seal character recognition model, which includes an encoder and two parallel sub-network branches; the encoder parses the small seal characters pictures into high-dimensional feature vectors; the two parallel sub-network branches include a "small seal character-traditional character conversion generation network" branch and an "attention-based Chinese character root structure recognition network" branch; branch one includes two decoders, which obtain the corresponding traditional character pictures and small seal characters according to the high-dimensional feature vectors to assist in the generation of the high-dimensional feature vectors of small seal characters; branch two obtains the Chinese character root structure vector of small seal characters according to the high-dimensional feature vectors, thereby recognizing small seal characters; finally, set the model loss function so that the two branches of the model assist and restrict each other, and obtain the final small seal character recognition model; The construction of the Xiaozhuan-Traditional Chinese conversion generation network branch in the Xiaozhuan recognition model is as follows: The Xiaozhuan-Traditional conversion generation network branch is composed of two decoders G1 and G2 with completely identical structures and two corresponding discriminators D1 and D2 with completely identical structures; G1 is used to decode and generate the corresponding Traditional Chinese picture according to the high-dimensional feature vector generated by the encoder E, and D1 is used to distinguish the input Traditional Chinese picture and the generated Traditional Chinese picture as a real image or a false image, and feed the discrimination result back to the encoder E, so as to improve the picture decoding quality of the encoder E and the Traditional Chinese picture generation quality of G1; G2 and D2 have similar functions to G1 and D1, and are used to generate Xiaozhuan pictures according to the high-dimensional feature vector and to judge the quality of Xiaozhuan pictures respectively; The attention-based Chinese character root structure recognition network branch adopts the Transformer decoding layer structure; the data2 feature data with dimensions of 16, 4, 4 and 512 generated by the encoder E is reshaped into 16, 16 and 512 dimensions as network input; the step size of the Transformer decoding layer is set to 16, the dimension is 512, and the prediction categories are 409 in total, of which 4 are mask marks and 405 are the number of font root categories; finally, the probabilities of 16 Chinese character root category predictions are output, and the position with the largest probability is taken as the root prediction result of the step size, and finally a feature vector representing the semantic information of the Chinese character root structure is obtained; the above-mentioned Chinese character root category prediction results are compared with the IDS dictionary using the K nearest neighbor method to obtain the final prediction result.

2. The zero-sample small seal recognition method based on Chinese character root structure according to claim 1 is characterized in that The construction of the training set and the test set is as follows: 1-1. Select N small seal characters T with similar structural contents to their corresponding traditional characters from the small seal character library, sort the small seal character set T according to the shape structure of each small seal character and the number of occurrences of its radicals, select N1 small seal characters T1 so that the number of occurrences of each radical in T1 is as uniform as possible, and remove the remaining N2 characters in T from the T1 character set as the character set T2; 1-2. Generate N1 small seal characters images with font A1 and content from character set T1, where A1 is the "Founder Small Seal Character" font; Generate corresponding traditional Chinese characters images with font B and content from character set T1, where B is the "Founder Handwriting - European Traditional Character" font, and splice N1 small seal characters images with font A1 of the character set T1 and the corresponding traditional Chinese characters images with font B one by one as the training set TR; 1-3. Build test sets TE1, TE2, TE3: N2 pictures of small seal characters in A1 font and pictures of traditional Chinese characters in B font of the character set T2 are correspondingly spliced ​​as training set TE1; N21 pictures of small seal characters in A2 font and pictures of traditional Chinese characters in B font of the character set T1 are correspondingly spliced ​​as training set TE2, wherein A2 is the "Han Yi Zhuan Shu Fan" font; N2 pictures of small seal characters in A2 font and pictures of traditional Chinese characters in B font of the character set T2 are correspondingly spliced ​​as training set TE3; 1-4. Collect the shape structure and radical structure of the traditional Chinese characters contained in the character set T, and encode all the shape structures and Chinese character radicals, each shape structure and each Chinese character radical has a unique code, find the shape structure and radical corresponding to the traditional Chinese characters in the character set T, convert each shape structure and radical into its corresponding coding sequence, add the numbers 2 and 3 at the beginning and end of the code as the label start and end marks of the text recognition network, and finally use 0 at the end of the sequence to fill the coding sequence to a length of 17 labels as the predicted label for each data set image.

3. The zero-sample small seal recognition method based on Chinese character root structure according to claim 1 is characterized in that The encoder E in the Xiaozhuan recognition model is constructed as follows: The encoder E is composed of eight convolution modules e1~e8, the convolution kernel size of the e1~e8 convolution modules is set to 2×2, the step size is set to 2, and the padding is set to SAME; according to the direction of forward propagation, the number of convolution kernels of the e1~e8 convolution modules is set to 64, 128, 256, 512, 512, 512, 512 and 512 respectively; and a layer of LReLu mapping is added before each convolution module of e2~e8 to complete the nonlinear transformation of the data, and a batch normalization layer BN is added after each convolution module of e2~e8; the feature array after processing by the e8 layer is called data1, and the feature array after processing by the e6 layer is called data2.

4. The zero-sample small seal recognition method based on Chinese character root structure according to claim 1 is characterized in that The decoder includes eight deconvolution modules g1~g8, the convolution kernel size of the g1~g8 convolution modules is set to 2×2, the step size is set to 2, and the number of convolution kernels of the g1~g8 convolution modules is set to 512, 512, 512, 512, 512, 256, 128 and 64 respectively according to the direction of forward propagation; a layer of ReLU mapping is added before each deconvolution module of g1~g8 to complete the nonlinear transformation of the data, and a BN layer is added after each deconvolution module of g1~g7. After BN, each deconvolution module of g1~g7 is connected with the e7~e1 modules in the encoder E respectively, g1~g3 needs to perform a dropout operation before the concat operation, and the probability is set to 0.

5. Finally, a tanh activation operation is performed on the g8 deconvolution module structure, so that the model converges quickly.

5. The zero-sample small seal recognition method based on Chinese character root structure according to claim 1 is characterized in that The discriminator includes 4 convolution modules d1~d4, each of which adopts the structure in CNN, the convolution kernel size of the d1~d4 convolution modules is set to 2×2, the step size is set to 2, and the padding is set to SAME; according to the direction of forward propagation, the number of convolution kernels of the d1~d4 convolution modules is set to 64, 128, 256 and 512 respectively; a layer of LReLU mapping is added after each convolution module of d1~d4, and a BN layer is added after the LReLU mapping of the d2~d4 convolution modules, and a fully connected layer is added after the d4 convolution module to judge whether the image is true or false.

6. The zero-sample small seal recognition method based on Chinese character root structure according to claim 1 is characterized in that The loss function of the seal character recognition model is designed as follows: The Xiaozhuan-Traditional conversion generation network branch mainly includes two parts: the discriminator loss and the generator loss. The discriminator loss is used to distinguish the authenticity of the generated image and the original image, wherein the D1 discriminator is used to distinguish the authenticity of the Traditional Chinese image, and the D2 is used to distinguish the authenticity of the Xiaozhuan image. The G1 and G2 losses consist of three parts: cycle consistency loss, feature matching loss, and generative adversarial network loss; the calculation formula of the target loss function combining the three is: L(X1,G(x),D(x),x,y)=αL p (G(x),y)+βL c (E(x))+L G (D(G(x))) Among them, x is the input image to be identified for generation, y is the target image, X1 is the high-dimensional feature vector, G(x) is the generated image, and D(x) is the discrimination result; α and β are L p , L c The weight value, L p To determine the cycle consistency loss, make the generated image as consistent as possible with the target image, L c For feature loss matching, the high-dimensional features of small seal characters and traditional characters are consistent. G is the loss function of the generative adversarial network, L G Least squares loss is adopted; In order to make full use of the semantic information of the similar structure between Xiaozhuan and traditional Chinese characters, the feature vectors of data1 obtained by encoder E encoding Xiaozhuan and traditional Chinese images should be as consistent as possible; at the same time, data2 is used as the input of the attention-based Chinese character root structure recognition network branch, and its output feature vectors should be as consistent as possible. c The description is as follows: L c =||Data2(x1)-Data2(x2)||1+||Data1(x1)-Data1(x2)||1 Wherein, Data2 represents the data2, Data1 represents the data1, x1 and x2 represent the input seal characters and traditional characters images respectively; The attention-based Chinese character root structure recognition network branch uses the normalized exponential function cross entropy as the loss function to calculate the consistency loss between the output semantic feature vector and the traditional Chinese character root structure vector.

Images