Zero-sample Chinese character recognition method based on radical representation learning

Abstract

The invention provides a radical representation learning-based zero-sample Chinese character recognition method, which comprises the following steps of: establishing a zero-sample Chinese character recognition data set, and obtaining an ideographic description sequence of Chinese characters; obtaining semantic embedding vectors of all Chinese characters and all radical and structure embedding vectors according to the ideographic description sequence; extracting image feature vectors of the Chinese character images through ResNet; the image feature vectors are processed by a feature enhancement coding module to obtain enhanced features; a radical representation decoding module converts the enhanced features into radical representation features under the guidance of radical and structure embedding vectors; and the Chinese character matching module predicts a semantic embedding vector by using the radical representation features and the radical and structure embedding vector, and matches with all semantic vectors to identify Chinese characters. According to the method, the problem of entanglement of geometric features of an image region is relieved, the transferability of image features to semantic features is improved, and the recognition capability of unseen Chinese characters is effectively improved by utilizing radical embedding to learn and position key radical representation in a Chinese character image.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and pattern recognition, and in particular to a zero-shot Chinese character recognition method based on radical representation learning. Background Art

[0002] Chinese character recognition has a long research background and is widely used. However, traditional Chinese character recognition models are based on the closed-set assumption and cannot recognize Chinese character categories not seen in the training set. In the current era of the prevalence of the Internet, new characters and rare characters are emerging continuously, making Chinese character recognition face severe challenges. In recent years, zero-shot Chinese character recognition has received extensive research attention because of its ability to recognize unseen Chinese character classes, such as historical document recognition and open-set text recognition. Zero-shot Chinese character recognition changes the recognition idea of traditional Chinese character recognition. It regards Chinese characters as a more fine-grained object (description of radicals or strokes), rather than a whole (character). Since radicals and strokes are shared attributes of Chinese characters, zero-shot Chinese character recognition methods can explore and learn the attribute knowledge of seen character classes, so as to have the ability to recognize unseen character classes. Existing zero-shot Chinese character recognition methods attempt to decompose Chinese characters into a sequence form, predict their radical or stroke sequences by decoding visual features through attention, and then predict Chinese characters through a matching algorithm (such as edit distance). According to the smallest unit of the sequence, they can be divided into radical-level and stroke-level methods. However, since the predicted sequences often have repetitions, omissions and errors, the method based on sequence decomposition has a very low error tolerance. To solve this problem, some methods propose to construct embedding vectors for radicals according to the ideographic description sequence (IDS), and then perform character recognition by transforming image features into the same semantic space.

[0003] Although these methods have gradually improved the defects of zero-shot Chinese character recognition, they directly decode the global image features, and these features are not sufficient to represent the fine-grained information of Chinese characters (such as the radical "讠" of "说"), because this key radical is only contained in a small part of the visual image. Therefore, the transformation of image features directly to semantic features is defective, resulting in poor visual-semantic interaction. On the other hand, most of these IDS embedding methods are manually designed, the information representing Chinese characters is extremely limited, and the spatial vectors are sparse, resulting in easy errors in visual-semantic interaction. Summary of the Invention

[0004] The purpose of the present invention is to propose a zero-shot Chinese character recognition method based on radical representation learning, which enhances the regional feature correlation to improve its transformability. By locating the discriminative radical region of image features to learn more fine-grained radical representation features, the reliability of the transformation of image features to semantic features is improved, effectively solving the model defects in the zero-shot setting and improving the recognition ability of unseen Chinese characters.

[0005] To achieve the above object, the technical solution of the present invention is: a zero-sample Chinese character recognition method based on radical representation learning, comprising the following steps;

[0006] Step S1, establish a zero-sample Chinese character recognition data set, parse the Chinese characters in the Chinese character recognition data set into a unique radical binary tree structure according to the radical and structure, obtain the Chinese character semantic description sequence by traversing the radical binary tree in pre-order, and set the semantic description sequence of Chinese character c to be IDS(c) = r1, r2, r3, ..., r n , where r1 to r n Indicates the category of radical or structure indicator, and n indicates the number of radicals and structure indicators in the semantic description sequence;

[0007] Step S2: Build the semantic embedding vectors of all Chinese characters based on the semantic description sequences of all Chinese characters in the dataset. and the embedding vectors of all radicals and structures Where N represents the total number of Chinese character categories, M represents the number of all radicals and structural symbols contained in all Chinese characters, and C represents the number of channels;

[0008] Step S3: The Chinese character image is passed through the ResNet network to extract the deep features of the image. The deep features of the Chinese character image are expressed as Where H, W and C represent the height, width and number of channels of the feature map respectively;

[0009] Step S4, establish a feature enhancement coding module, which represents the depth feature F of the Chinese character image in grid form, obtains the relative geometric features of the Chinese character image by calculating the relative relationship between each grid feature, and thus enhances the features of the Chinese character image, and records the enhanced features as During the enhancement process, the height and width of the Chinese character image features are flattened, that is, I = H·W;

[0010] Step S5: Establish a radical representation decoding module. Under the guidance of the radical and structure embedding vector A, the radical representation decoding module is The most critical radical representation features in Chinese character images are learned and located, and the image features represented by radicals are recorded as

[0011] Step S6: Establish a Chinese character matching module. The Chinese character matching module uses the image features F represented by radicals. R and the embedding vector A of the radical and structure to generate the predicted semantic embedding vector of the Chinese character image Then S pred Match it with the semantic embedding vector S of all Chinese characters and select the result with the highest probability as the recognition result.

[0012] Preferably, in step S1, the leaf nodes of the radical binary tree are all radical indicators, the parent nodes are all structure indicators, and the child nodes are composed of Chinese characters according to the structure of the parent nodes.

[0013] Preferably, in step S2, the semantic embedding vector of the Chinese character is obtained according to the semantic description sequence of the Chinese character, and its embedding expression is:

[0014]

[0015] Among them, r i ∈IDS(c) represents the category of the oi-th radical or structural indicator in the ideographic description sequence of Chinese character c, p(r i ) represents r i The probability of appearing in all Chinese character radicals and structures.

[0016] Preferably, the embedding vectors of radicals and structures are obtained by embedding the Word2Vec model, and the embedding expression is:

[0017]

[0018] in, The set of all radicals and structural indicators obtained by counting the semantic description sequences of all Chinese characters, M represents the number of all radicals and structural indicators, Indicates a radical or a structural indicator.

[0019] Preferably, in step S4, the relative geometric features are obtained by:

[0020] The Chinese character image depth feature F is grid-encoded and divided into H·W grids according to the height and width of the feature map. The 2D position coordinate pair of each grid is obtained. The 2D position coordinate pair of the pth grid is set to Calculate the center coordinates of the pth grid through the 2D position coordinate pair (cu p ,cv p ) and the width w of the grid p and high h p :

[0021]

[0022] in, and Respectively represent the coordinates of the upper left corner and lower right corner of the grid;

[0023] Then the regional geometric characteristics G of the pth grid relative to the qth grid are pq It is expressed as:

[0024]

[0025] Among them, ReLU() represents the activation function, FC() represents the linear layer, represents the learnable hyperparameter matrix, R pq Indicates the relative geometric relationship between the p-th grid and the q-th grid;

[0026] Based on the above formula, in order to characterize the global geometric relationship of the entire Chinese character image, the final relative geometric feature G is obtained by taking the average value of the relative geometric features between all grids, as follows:

[0027]

[0028] Among them, Z represents the total number of grids on the feature map, G pq Represents the relative geometric characteristics of the p-th grid relative to the q-th grid.

[0029] Preferably, in step S4, the enhanced feature Based on the extracted relative geometric features G, the scaled dot product attention is used to enhance the deep features F of the Chinese character image to obtain the refined enhanced features The details are as follows:

[0030]

[0031]

[0032] Among them, Q F , K F and V F Represent the query, key, and value in the attention calculation, respectively. and denote the learnable hyperparameter matrices in query, key, and value, respectively, Z avg represents the attention feature minus the geometric entanglement, softmax() is the normalized exponential function, represents the scale factor of attention. Formula (10) represents the enhanced feature obtained by adding the attention feature minus the geometric entanglement to the Chinese character image depth feature F

[0033] Preferably, the step S5 is specifically:

[0034] The radical decoding module is designed using the Transformer network, which uses the embedded vector A of radicals and structures to learn enhanced features. The radical representation in the figure, the embedded vector A of the radical and structure is used as the query of the multi-head attention layer to enhance the features As keys and values, parse the enhanced features to get multi-head attention features The calculation process of the multi-head attention mechanism is expressed as:

[0035]

[0036] Among them, Q j , K j and V j denote the query, key, and value of the jth attention head, respectively. and represents the learnable hyperparameter matrix of query, key, and value in the jth attention head, head j represents the jth attention head, d k is the scaling factor, represents the multi-head attention feature, represents the feature concatenation operation, W o represents a matrix of learnable hyperparameters;

[0037] The obtained multi-head attention features will enter the feed-forward layer of the radical decoding module to generate the radical-represented image features F R :

[0038]

[0039] Among them, ReLU() represents the activation function, W1 and W2 represent learnable hyperparameters, and bias1 and bias2 represent bias values.

[0040] Preferably, in step S6, the predicted semantic embedding vector S pred The generation of and the acquisition of the recognition result are specifically as follows:

[0041] S pred =A T W s F R (15)

[0042]

[0043] Among them, A T The transpose of the embedding vector A representing the radical and structure, W s is the auxiliary embedding matrix, y represents the predicted Chinese character label, Y all =Y U ∩Y S is the total Chinese character set, including the unseen Chinese character set Y U and the set of seen Chinese characters Y S .

[0044] Preferably, the ResNet network, feature enhancement encoding module, radical representation decoding module and Chinese character matching module mentioned in steps S3, S4, S5 and S6 adopt the radical embedding cross entropy function as a whole. Perform parameter optimization:

[0045]

[0046] in, represents the predicted semantic vector of the lth image sample in the batch, N B Indicates the number of training samples in a batch during the training process. represents the true semantic embedding vector of the corresponding batch, S y Semantic embedding vector representing opinion categories.

[0047] Preferably, the ResNet network, feature enhancement encoding module, radical representation decoding module and Chinese character matching module mentioned in steps S3, S4, S5 and S6 adopt radical regression loss as a whole. Optimize, radical regression loss Parameter optimization is achieved by minimizing the mean squared error between the true semantic embedding vector and the predicted vector:

[0048]

[0049] in, represents the predicted semantic vector of the lth image sample in the batch, Represents the true semantic embedding vector of the corresponding batch; express and Calculate the mean square error.

[0050] The present invention proposes a zero-shot Chinese character recognition method based on radical representation learning, which enhances the correlation of regional features to improve its convertibility. By locating areas where visual features are discriminative, a more fine-grained radical representation is learned, thereby improving the reliability of visual to semantic conversion. Specifically, the feature enhancement coding module subtracts the entangled features of the flattened image feature area to improve the transferability of image characteristics to semantic features. In order to learn a more fine-grained radical representation, the present invention processes the radicals and structures into word vector forms, and locates the image areas most relevant to each radical and structure in a given image under the guidance of the radical representation through a radical representation decoder. Finally, recognition is achieved using visual features embedded with radical representations and IDS embedding. This method can effectively solve the shortcomings of model recognition under zero-shot settings and improve the recognition performance of unseen Chinese characters.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention reduces the entanglement problem of image features by utilizing the regional geometric embedding features of the image through a feature enhancement coding module to enhance the convertibility of image features; improves recognition performance by learning to locate the image region most relevant to each radical and structure in the image under the guidance of the radical and structure embedding vector through a radical representation decoding module; and effectively predicts the semantic vector of the Chinese character image by combining the radical representation features with the radical and structure embedding vector through a Chinese character matching module, thereby achieving effective Chinese character recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flow chart of the method of the present invention;

[0054] Figure 2 A schematic diagram of a sequence of Chinese character meaning descriptions of the present invention;

[0055] Figure 3 It is a schematic diagram of the radical knowledge of the present invention;

[0056] Figure 4 This is a schematic diagram of the network structure of the present invention. DETAILED DESCRIPTION

[0057] The following is combined with Figure 1-4 , the technical solution of the present invention is specifically described.

[0058] The present invention proposes a zero-sample Chinese character recognition method based on radical representation learning, comprising the following steps:

[0059] Step S1, establishing a zero-sample Chinese character recognition data set and obtaining a semantic description sequence of Chinese characters;

[0060] Step S2, using the semantic description sequence to obtain the semantic embedding vector and radical and structure embedding vector of the Chinese character;

[0061] Step S3, extracting feature vectors of Chinese character images;

[0062] Step S4, constructing an image feature vector and passing it through a feature encoding module to obtain enhanced characteristics;

[0063] Step S5, the radical representation decoding module converts the enhanced features into radical representation features using the radical and structure embedding vector;

[0064] Step S6: The Chinese character matching module predicts the semantic embedding vector of the image using the radical representation feature and the radical and structure embedding vector, and performs Chinese character matching.

[0065] In step S1, Chinese characters are decomposed into a radical binary tree according to radicals and structures. The characteristics are that the leaf nodes are all radical symbols, the parent nodes are all structure representation symbols, and the child nodes are composed of Chinese characters according to the structure of the parent nodes. Since the radical and structure are common features between Chinese characters, they are used as common attributes between the seen Chinese characters and the unseen Chinese characters.

[0066] In step S2, the semantic embedding vector of the Chinese character will be used as the label of the Chinese character to participate in the recognition of the model. Each Chinese character category has a unique semantic embedding vector to represent it. The semantic embedding vector of the Chinese character is obtained according to the ideographic descriptor of the Chinese character, and its embedding formula can be expressed as:

[0067]

[0068] Among them, r i ∈IDS(c) represents the ith radical or structural ideographic character in the Chinese character ideographic description sequence, p(r i ) represents r i The probability of appearing in all Chinese character radicals and structures;

[0069] The embedded vectors of radicals and structures will be used as labels for radical and structure indicators in the model prediction. Each radical and structure indicator has a unique word vector to represent it. By counting the semantic description sequences of all Chinese characters, the set of all radicals and structure indicators can be obtained as Where M represents the number of all radicals and structure indicators. Then, the embedding vectors of radicals and structures are obtained by embedding the Word2Vec model. The embedding formula can be expressed as:

[0070]

[0071] in, Indicates a radical or a structural indicator.

[0072] In step S4, in order to calculate the relative geometric relationship of the depth features of the Chinese character image, the feature enhancement module needs to grid-code the depth features F of the Chinese character image. According to the height and width of the feature map, it can be divided into H·W grids, and then the 2D position coordinate pair of each grid can be obtained. Let the 2D position coordinate pair of the pth grid be The center coordinates of the pth grid can be calculated through the 2D position coordinate pair (cu p ,cv p ) and the width w of the grid p and high h p :

[0073]

[0074] in, and Represents the coordinates of the upper left corner and lower right corner of the grid respectively.

[0075] In step S4, the relative geometric relationship between the grids is established. Given the pth and qth grids, the regional geometric feature G of the pth grid relative to the qth grid is pq It can be expressed as:

[0076]

[0077] Among them, ReLU() represents the activation function, FC() represents the linear layer, represents the learnable hyperparameter matrix, R pq represents the relative geometric relationship between the p-th grid and the q-th grid, which can be calculated by formula (7);

[0078] Based on the above formula, in order to characterize the global geometric relationship of the entire Chinese character image, the final relative geometric feature g is obtained by taking the average value of the relative geometric features between all grids, as follows:

[0079]

[0080] Among them, Z represents the total number of grids on the feature map, G pq Represents the relative geometric characteristics of the p-th grid relative to the q-th grid.

[0081] In step S4, the relative geometric feature G of the entire image can be extracted from the Chinese character image deep feature F, and the Chinese character image deep feature F can be enhanced using scaled dot product attention to obtain a refined enhanced feature The above process can be expressed as follows:

[0082]

[0083] Among them, Q F , K F and V F Represent the query, key, and value in the attention calculation, respectively. and denote the learnable hyperparameter matrices in query, key, and value, respectively, Z avg represents the attention feature minus the geometric entanglement, softmax() is the normalized exponential function, represents the scaling factor of attention, and formula (10) represents the attention feature Z minus the geometric entanglement avg Added to the Chinese character image depth feature F to obtain enhanced features

[0084] In step S5, the radical decoding module is designed using the Transformer network, and the embedded vector A of the radical and structure is used to learn the enhanced features The radical representation in the figure, the embedded vector A of the radical and structure is used as the query of the multi-head attention layer to enhance the features As keys and values, parse the enhanced features to get multi-head attention features The calculation process of the multi-head attention mechanism can be expressed as:

[0085]

[0086]

[0087] Among them, Q j , K j and V j denote the query, key, and value of the jth attention head, respectively. and represents the learnable hyperparameter matrix of query, key, and value in the jth attention head, head j represents the jth attention head, d k is the scaling factor, represents the multi-head attention feature, represents the feature concatenation operation, W o represents a matrix of learnable hyperparameters.

[0088] In step S5, the multi-head attention features will enter the feed-forward layer of the radical decoding module to generate the radical-represented image features F R , the process can be expressed as:

[0089]

[0090] Among them, ReLU() represents the activation function, W1 and W2 represent learnable hyperparameters, and bias1 and bias2 represent bias values.

[0091] In step S6, the Chinese character matching module uses the image feature F represented by the radical R The semantic embedding vector S of the image is predicted by combining the embedding vector A of the radical and structure pred , and then match with all Chinese character semantic embedding vectors to calculate the probability. The above process can be expressed as follows:

[0092] S pred =A T W s F R (15)

[0094]

[0095] Among them, A T The transpose of the embedding vector A representing the radical and structure, W s is the auxiliary embedding matrix, y represents the predicted Chinese character label, Y all =Y U ∩Y S is the total Chinese character set, which includes the unseen Chinese character set Y U and the set of seen Chinese characters Y S .

[0096] The multiple modules mentioned in steps S3, S4, S5 and S6 are embedded in the cross entropy loss function through radicals. Radical regression loss function To optimize

[0097] In order to make the predicted semantic embedding vector closer to the real image distribution and obtain a better compatibility score, the present invention adopts the radical embedding cross entropy function To optimize the model:

[0098]

[0099]

[0100] in, represents the predicted semantic vector of the lth image sample in the batch, N B Indicates the number of training samples in a batch during the training process. represents the true semantic embedding vector of the corresponding batch, S y Semantic embedding vector representing opinion category;

[0101] Radical regression loss The model is optimized by minimizing the mean squared error between the true semantic embedding vector and the predicted vector:

[0102]

[0103] in, represents the predicted semantic vector of the lth image sample in the batch, Represents the true semantic embedding vector of the corresponding batch; express and Calculate the mean square error.

[0104] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A zero-shot Chinese character recognition method based on radical representation learning, characterized in that: The steps include: Step S1, establish a zero-sample Chinese character recognition data set, parse the Chinese characters in the Chinese character recognition data set into a unique radical binary tree structure according to the radical and structure, obtain the Chinese character semantic description sequence by traversing the radical binary tree in pre-order, and set the semantic description sequence of Chinese character c to be recorded as IDS(c)=r1,r2,r3,...,r n , where r1 to r n Indicates the category of radical or structure indicator, and n indicates the number of radicals and structure indicators in the semantic description sequence; Step S2: Build the semantic embedding vectors of all Chinese characters based on the semantic description sequences of all Chinese characters in the dataset. and the embedding vectors of all radicals and structures Where N represents the total number of Chinese character categories, M represents the number of all radicals and structural symbols contained in all Chinese characters, and C represents the number of channels; Step S3: The Chinese character image is passed through the ResNet network to extract the deep features of the image. The deep features of the Chinese character image are expressed as Where H, W and C represent the height, width and number of channels of the feature map respectively; Step S4, establish a feature enhancement coding module, which represents the depth feature F of the Chinese character image in grid form, obtains the relative geometric features of the Chinese character image by calculating the relative relationship between each grid feature, and thus enhances the features of the Chinese character image, and records the enhanced features as During the enhancement process, the height and width of the Chinese character image features are flattened, that is, I = H·W; Step S5: Establish a radical representation decoding module. Under the guidance of the radical and structure embedding vector A, the radical representation decoding module is The most critical radical representation features in Chinese character images are learned and located, and the image features represented by radicals are recorded as Step S6: Establish a Chinese character matching module. The Chinese character matching module uses the image features F represented by radicals. R and the embedding vector A of the radical and structure to generate the predicted semantic embedding vector of the Chinese character image Then S pred Match it with the semantic embedding vector S of all Chinese characters and select the result with the highest probability as the recognition result.

2. A zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: In step S1, the leaf nodes of the radical binary tree are all radical indicators, the parent nodes are all structure indicators, and the child nodes are composed of Chinese characters according to the structure of the parent nodes.

3. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: In step S2, the semantic embedding vector of the Chinese character is obtained according to the semantic description sequence of the Chinese character, and its embedding expression is: Among them, ri∈IDS(c) represents the category of the ith radical or structural indicator in the ideographic description sequence of Chinese character c, and p(ri) represents r i The probability of appearing in all Chinese character radicals and structures.

4. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: The embedding vectors of radicals and structures are obtained by embedding the Word2Vec model, and their embedding expressions are: in, The set of all radicals and structural indicators obtained by counting the semantic description sequences of all Chinese characters, M represents the number of all radicals and structural indicators, Indicates a radical or a structural indicator.

5. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: In step S4, the relative geometric features are obtained as follows: The Chinese character image depth feature F is grid-encoded, and it is divided into HW grids according to the height and width of the feature map. The 2D position coordinate pair of each grid is obtained. The 2D position coordinate pair of the pth grid is set to Calculate the center coordinates of the pth grid through the 2D position coordinate pair (cu p , cv p ) and the width w of the grid p and high h p : in, and Respectively represent the coordinates of the upper left corner and lower right corner of the grid; Then the regional geometric characteristics G of the pth grid relative to the qth grid are pq It is expressed as: Among them, ReLU( ) represents the activation function, FC( ) represents the linear layer, represents the learnable hyperparameter matrix, R pq Indicates the relative geometric relationship between the p-th grid and the q-th grid; The relative geometric feature G is obtained by taking the average value of the relative geometric features between all grids, as follows: Among them, Z represents the total number of grids on the feature map, G pq Represents the relative geometric characteristics of the p-th grid relative to the q-th grid.

6. A zero-shot Chinese character recognition method based on radical representation learning according to claim 5, characterized in that: In step S4, the enhanced feature Based on the extracted relative geometric features G, the scaled dot product attention is used to enhance the deep features F of the Chinese character image to obtain the refined enhanced features The details are as follows: Among them, Q F , K F and V F Represent the query, key, and value in the attention calculation, respectively. and denote the learnable hyperparameter matrices in query, key, and value, respectively, Z avg represents the attention feature minus the geometric entanglement, softmax() is the normalized exponential function, represents the scale factor of attention. Formula (10) represents the enhanced feature obtained by adding the attention feature minus the geometric entanglement to the Chinese character image depth feature F 7. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: The step S5 is specifically as follows: The radical decoding module is designed using the Transformer network. The embedded vector A of the radical and structure is used to learn the radical representation in the enhanced feature F. The embedded vector A of the radical and structure is used as the query of the multi-head attention layer to enhance the feature As keys and values, parse the enhanced features to get multi-head attention features The calculation process of the multi-head attention mechanism is expressed as: Among them, Q j , K j and V j denote the query, key, and value of the jth attention head, respectively. and represents the learnable hyperparameter matrix of query, key, and value in the jth attention head, head j represents the jth attention head, d k is the scaling factor, represents the multi-head attention feature, represents the feature concatenation operation, W o represents a matrix of learnable hyperparameters; The obtained multi-head attention features will enter the feed-forward layer of the radical decoding module to generate the radical-represented image features F R : Among them, ReLU() represents the activation function, W1 and W2 represent learnable hyperparameters, and bias1 and bias2 represent bias values.

8. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: In step S6, the predicted semantic embedding vector S pred The generation of and the acquisition of the recognition result are specifically as follows: S pred =A T W s F R (15) Among them, A T The transpose of the embedding vector A representing the radical and structure, W s is the auxiliary embedding matrix, y represents the predicted Chinese character label, Y all =Y U ∩Y S is the total Chinese character set, including the unseen Chinese character set Y U and μ see Chinese character set Y S .

9. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: The ResNet network, feature enhancement encoding module, radical representation decoding module and Chinese character matching module mentioned in steps S3, S4, S5 and S6 adopt the radical embedding cross entropy function as a whole. Perform parameter optimization: in, represents the predicted semantic vector of the lth image sample in the batch, N B Indicates the number of training samples in a batch during the training process. represents the true semantic embedding vector of the corresponding batch, S y Semantic embedding vector representing opinion categories.

10. The zero-shot Chinese character recognition method based on radical representation learning according to claim 1, characterized in that: The ResNet network, feature enhancement encoding module, radical representation decoding module and Chinese character matching module mentioned in steps S3, S4, S5 and S6 adopt the radical regression loss as a whole. Optimize, radical regression loss Parameter optimization is achieved by minimizing the mean squared error between the true semantic embedding vector and the predicted vector: in, represents the predicted semantic vector of the lth image sample in the batch, Represents the true semantic embedding vector of the corresponding batch; express and Calculate the mean square error.

Images