A zero-shot Chinese character recognition method based on diverse feature generation
Generating diversified features through mixed semantic embedding, the problem of relying on unstable radicals or stroke prediction in the prior art is solved, the robustness and fault tolerance of zero-sample Chinese character recognition are improved, and more efficient recognition performance is achieved.
Patent Information
- Application Number
- CN202410808562.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-21
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-06-21
AI Technical Summary
The existing zero-sample Chinese character recognition method relies on unstable radicals or stroke prediction, resulting in low recognition performance, and the problems of unbalanced training data and radical radicals are difficult to solve.
Using a method based on diverse feature generation, the mixed semantic embedding of radicals from information and word vectors is generated to generate diverse training samples of unknown classes to avoid dependence on radicals or strokes, and the feature generation network is used to generate more efficiently. Robust features, used to train and recognize models.
It improves the fault tolerance and robustness of zero-sample Chinese character recognition, enhances the diversity and generalization ability of feature generation, and avoids the decrease in recognition accuracy caused by errors in radical or stroke prediction.
Smart Images

Figure CN118378621B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision and pattern recognition, and in particular to a zero-sample Chinese character recognition method based on diverse feature generation. Background Art
[0002] Chinese character recognition has made many important advances due to the development of deep learning. However, in reality, we often need to deal with many uncommon Chinese characters, and it is almost impossible to collect and annotate them all. As a result, more and more research has begun to focus on zero-shot Chinese character recognition, such as open set text recognition and historical document recognition, trying to explore a more robust pattern recognition method through zero-shot learning.
[0003] Zero-shot Chinese character recognition usually adopts the radical or stroke of Chinese characters as an intermediate representation between the seen Chinese character class and the unseen Chinese character class. These methods match the target Chinese character by encoding and decoding the input image into a stroke or radical representation. Since strokes and radicals are shared attributes between Chinese characters, these methods can learn knowledge of intermediate representations from the seen Chinese character class, enabling them to recognize the unseen Chinese character class. Although recent zero-shot Chinese character recognition methods have made significant progress, the performance is still far from the needs of practical applications. A major problem with most existing methods is that the recognition performance of current methods largely depends on the prediction of radicals or strokes, which is unreliable and error-prone, which deteriorates the performance of subsequent matching or recognition. At the same time, some open issues, such as imbalanced training data and zero-shot problems of radicals, also need further study. Summary of the invention
[0004] The present invention proposes a zero-shot Chinese character recognition method based on diverse feature generation, which avoids the reliance on radical or stroke prediction, thereby enhancing the fault tolerance and robustness of recognition. On the other hand, in order to improve the diversity of generated features, we designed a hybrid semantic embedding method. This method enhances the semantic expression of feature generation by hierarchical decomposition of the Chinese character semantic description sequence, radical self-information, and word vector, so that the generated features can be directly used to train the recognition model.
[0005] The present invention adopts the following technical scheme: A zero-shot Chinese character recognition method based on diverse feature generation is intended to utilize the semantic embedding information of Chinese character radicals to generate diverse unseen class training samples for zero-shot Chinese character recognition tasks, comprising the following steps;
[0006] Step S1: By representing the radical and structure of each Chinese character as a tree structure, the tree structure is traversed in pre-order to obtain the ideographic description sequence (IDS) of the Chinese character, which is expressed as , y represents the corresponding Chinese character category label; wherein, the ideographic description sequence is divided into a seen ideographic description sequence and an unseen ideographic description sequence according to whether it corresponds to a seen Chinese character image;
[0007] Step S2: embed the semantic description sequence of each Chinese character from three directions: hierarchical decomposition, radical self-information, and word vector to obtain three corresponding semantic vectors , and ;
[0008] Step S3: The seen Chinese character image is encoded into a feature map through the DenseNet169 network ,in , and Represent the height, width and number of channels of the feature map respectively, and then flatten it to get a The true characteristics of the dimension , and the true features of all the seen Chinese character images Constitute a real sample;
[0009] Step S4: Three semantic vectors of Chinese characters , and Assign a feature generation network FGN to learn each semantic vector to the real feature The mapping of
[0010] Step S5: Use the seen class samples to train the feature generation network FGN so that the FGN generates features Gradually approaching the true characteristics The distribution of the seen class samples includes the true features of all seen class Chinese character images , and the three semantic vectors of all the corresponding semantic description sequences , and ;
[0011] Step S6: Generate unseen class training feature samples by using the three semantic vectors obtained by embedding each unseen class semantic description sequence through the trained three feature networks FGN , and , obtain the unseen class training feature samples of all unseen class semantic description sequences , and As a generated sample set ;
[0012] Step S7: Generate sample set using and its corresponding Chinese character category labels, and the real samples and their corresponding Chinese character category labels together to train the recognition model so as to enable the recognition model to recognize Chinese character categories that have not been seen;
[0013] Step S8: extract features of the image to be identified through the DenseNet169 network and then input it into the trained recognition model to obtain the result.
[0014] Furthermore, in step S1, Chinese characters are decomposed into radicals and structural sequence representations to be used as common information between the seen Chinese characters and the unseen Chinese characters.
[0015] Furthermore, in step S2, the hierarchical decomposition embedding decomposes the Chinese character radical tree structure into For node representation, given a length of Node-path ,in, Represents the path connecting the nodes, and its influence value It is calculated by the following formula:
[0016] (1)
[0017] in, represents the influence coefficient that gradually decays as the node-path length increases, Indicates Path The influence coefficient of Indicates Path The influence coefficient of Indicates Path The influence value between the child nodes in the corresponding parent node; all nodes or node-paths in the radical tree are divided into two groups; one group is the leaf nodes representing the radical The set is represented by , and the other group is the parent node representing the structure The set is represented by ; Finally, the hierarchical embedding vector It is expressed as:
[0018] (2)
[0019] in, and Respectively represent the radical and structure in IDS, is a hyperparameter that balances leaf nodes and parent nodes, Represents a radical node The impact value of Represents a structure node The impact value of is calculated by formula (1);
[0020] Radical self-information embedding measures the key value of each radical from the perspective of information theory; then assign an index to each radical and structure and obtain its one-hot encoding, and obtain the radical self-information embedding vector through weighted encoding:
[0021] (3)
[0022] in, Indicated by IDS description sequence consisting of radicals and structures, Indicates the corresponding radical or structure after one-hot encoding, It is the probability distribution of radicals or structures, obtained by counting the IDS dictionary;
[0023] Word vector embedding uses the entire IDS dictionary as corpus to train a Word2vec model to obtain word embedding vectors for all radicals and structures; the average word embedding of the IDS sequence is taken as the embedding representation of each category:
[0024] (4)
[0025] in, Indicates radical or structure Word2Vec word embedding, L represents the length of IDS.
[0026] Furthermore, in step S3, the encoding network DenseNet169 first obtains a feature map through a convolution layer, and then performs a nonlinear transformation through a ReLU activation function; then, a series of dense blocks are used to gradually increase the dimension and complexity of the feature to obtain high-level abstract features of the image, remove the last classification layer, and take the flattened feature vector after average pooling as the real feature. Provides feature generation network learning.
[0027] Furthermore, in step S4, the feature generation network FGN input dimension is aligned with the corresponding semantic vector, and the feature generation network FGN includes a feature generation VAE and a feature generation WGAN, denoted as f-VAE and f-WGAN; f-VAE consists of an encoder Composition; f-WGAN consists of a generator and a discriminator The discriminator can distinguish the real features , and can also identify the generated features ; The f-VAE will utilize the generator in f-WGAN To assist in training f-VAE; the encoder The real features of the Chinese character image extracted by DenseNet169 are input Encoded into hidden state , and then by With real characteristics For the template Reconstruction to obtain generated features .
[0028] Furthermore, in step 4, the f-VAE is implemented through a VAE loss function Optimize to enable the encoder to better parse Chinese character image features and obtain more representative hidden states Vector representation,
[0029] (5)
[0030] in, is the KL divergence, which is used to measure the potential spatial distribution of the encoder output With the prior distribution The difference between , Indicates that the real feature and a semantic vector As an encoder Input and output hidden state ,in Used to indicate or or It means that in a given semantic vector Hidden state The prior distribution of It is the binary cross entropy used to calculate the real feature reconstruction loss, the hidden state of the encoder output The expected value of sampling, represent Relative to encoder expectations, among which Indicates that the hidden state and semantic vector As a generator The input of the generator The output of is the generated feature ; The generated features synthesized by the generator are minimized by the VAE loss function As close to the real features as possible .
[0031] Furthermore, in step 4, the Wasserstein GAN loss is introduced To constrain the generator so that the generator Correctly embed the semantic vector Reconstruct back to the real features :
[0032] (6)
[0033] in, Representation Discriminator For real features and semantic vector Expected rating, Representation Discriminator Generate features and semantic vector Expected rating of represents the gradient penalty term, which is used to satisfy the 1-Lipschitz constraint. f-WGAN requires the discriminator is 1-Lipschitz continuous, which means that for any input, the norm of its gradient should be close to 1; where In real characteristics and generate features The features obtained by random interpolation between Indicates that the discriminator is and The second norm of the gradient on , is a penalty term used to make the norm of the gradient close to 1. is the penalty coefficient, which is used to control the weight of the gradient penalty term;
[0034] For the generator Introduced a classification loss To generate features that are more suitable for classification, such as Figure 5 The corresponding classifier is shown, and its loss function is:
[0035] (7)
[0036] in, Represents generated features The Chinese character category labels predicted to be true The probability of Represents the probability distribution The generated features obtained by sampling The expected value of the conditional probability is given by the parameter The fully connected layer of the classifier is pre-trained based on the actual features of the categories seen;
[0037] The overall optimization function of the feature generation network is:
[0038] (8)
[0039] (9)
[0040] in and Yes Control and The hyperparameter of the loss weight, formula (9) represents the objective function optimized by the feature generation network FGN, where , and Represent the generator, encoder and discriminator respectively, and the optimization strategy is to minimize and while maximizing the loss loss.
[0041] Furthermore, in step S6, the semantic description sequence IDS of each unknown Chinese character is known, and its three semantic vectors are obtained, namely, , , ; The three semantic vectors are combined with random Gaussian noise Then they are input into the corresponding trained feature generation network FGN respectively. The three feature generation networks FGN generate corresponding unseen class training feature samples according to the semantic embedding vectors. , and Repeat the above feature generation process, traverse each unseen Chinese character IDS to obtain a generated sample set that mixes the three semantics .
[0042] Furthermore, the recognition model in step S7 is optimized by minimum negative log-likelihood loss:
[0043] (10)
[0044] in, Indicates that the real feature Map to The parameters of the fully connected layer of the Chinese character category are Optimize to minimize the objective function, that is , is the training dataset, Represents the true Chinese character category label calculated by softmax The probability distribution of is:
[0045] (11)
[0046] in, Is the real Chinese character category label The parameter vector and true feature The inner product of is the parameter vector and true feature of the mth Chinese character category The inner product of .
[0047] Furthermore, the recognition function of the recognition model trained in step S8 is:
[0048] (12)
[0049] in, Represents real characteristics, Represents the Chinese character category label.
[0050] The present invention proposes a zero-shot Chinese character recognition method based on diverse feature generation, which generates different features for unseen Chinese character classes through hybrid semantic embedding. The diversity and generalization ability of feature generation are improved by integrating multiple semantic information representations; a fully supervised recognizer is trained by generating unseen Chinese character class features through features, so that the recognition model can directly recognize Chinese characters without understanding the radical and stroke information in the image, fundamentally avoiding the reduction in recognition accuracy caused by incorrect prediction of radicals or strokes.
[0051] Compared with the prior art, the present invention has the following beneficial effects:
[0052] (1) The present invention provides a new idea for zero-sample Chinese character recognition, namely, using the radical description of Chinese characters to generate training features to alleviate the problem of lack of samples of unseen Chinese characters.
[0053] (2) The present invention does not rely on identifying radicals or strokes to recognize Chinese characters, thereby improving the error tolerance and robustness of recognition.
[0054] (3) The hybrid semantic embedding method proposed in this paper uses multiple semantic embeddings to represent a category, which helps to improve the diversity of synthetic features. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a flowchart diagram of the method of the present invention;
[0056] Figure 2 This is a schematic diagram comparing the generation of single semantic and multi-semantic features;
[0057] Figure 3 It is a comparative schematic diagram of the traditional framework and the framework of the present invention;
[0058] Figure 4 It is a diagram of the twelve radicals, tree structure and ideographic description sequence of Chinese characters;
[0059] Figure 5 It is a schematic diagram of the network structure of the present invention. DETAILED DESCRIPTION
[0060] like Figure 1-5 As shown, the present invention proposes a zero-sample Chinese character recognition method based on diverse feature generation, comprising the following steps:
[0061] Step S1: By representing the radical and structure of each Chinese character as a tree structure, the tree structure is traversed in pre-order to obtain the ideographic description sequence (IDS) of the Chinese character, which is expressed as , y represents the corresponding Chinese character category label; wherein, the ideographic description sequence is divided into a seen ideographic description sequence and an unseen ideographic description sequence according to whether it corresponds to a seen Chinese character image;
[0062] Step S2: embed the semantic description sequence of each Chinese character from three directions: hierarchical decomposition, radical self-information, and word vector to obtain three corresponding semantic vectors , and ;
[0063] Step S3: The seen Chinese character image is encoded into a feature map through the DenseNet169 network ,in , and Represent the height, width and number of channels of the feature map respectively, and then flatten it to get a The true characteristics of the dimension , and the true features of all the seen Chinese character images Constitute a real sample;
[0064] Step S4: Three semantic vectors of Chinese characters , and Assign a feature generation network FGN to learn each semantic vector to the real feature The mapping of
[0065] Step S5: Use the seen class samples to train the feature generation network FGN so that the FGN generates features Gradually approaching the true characteristics The distribution of the seen class samples includes the true features of all seen class Chinese character images , and the three semantic vectors of all the corresponding semantic description sequences , and ;
[0066] Step S6: Generate unseen class training feature samples by using the three semantic vectors obtained by embedding each unseen class semantic description sequence through the trained three feature networks FGN , and , obtain the unseen class training feature samples of all unseen class semantic description sequences , and As a generated sample set ;
[0067] Step S7: Generate sample set using and its corresponding Chinese character category labels, and the real samples and their corresponding Chinese character category labels together to train the recognition model so as to enable the recognition model to recognize Chinese character categories that have not been seen;
[0068] Step S8: extract features of the image to be identified through the DenseNet169 network and then input it into the trained recognition model to obtain the result.
[0069] In the step S1, Chinese characters are decomposed into radicals and structural sequence representations to be used as common information between the seen Chinese characters and the unseen Chinese characters.
[0070] In step S2, the hierarchical decomposition embedding decomposes the Chinese character radical tree structure into For node representation, given a length of Node-path ,in, Represents the path connecting the nodes, and its influence value It can be calculated by the following formula:
[0071] (1)
[0072] in, Indicates the influence coefficient that gradually decays as the node-path length increases. Take 1, 0.5, 0.1, 0.005, 0.001 and so on, Indicates Path The influence coefficient of Indicates Path The influence coefficient of Indicates Path The influence value between the child nodes in the corresponding parent node; all nodes or node-paths in the radical tree are divided into two groups; one group is the leaf nodes representing the radical The set is represented by , and the other group is the parent node representing the structure The set is represented by ; Finally, the layer embedding vector It is expressed as:
[0073] (2)
[0074] in, and Respectively represent the radical and structure in IDS, is a hyperparameter that balances leaf nodes and parent nodes, Represents a radical node The impact value of Represents a structure node The impact value of is calculated by formula (1);
[0075] Radical self-information embedding measures the key value of each radical from the perspective of information theory; then assign an index to each radical and structure and obtain its one-hot encoding, and obtain the radical self-information embedding vector through weighted encoding:
[0076] (3)
[0077] in, Indicated by IDS description sequence consisting of radicals and structures, Indicates the corresponding radical or structure after one-hot encoding, It is the probability distribution of radicals or structures, obtained by counting the IDS dictionary;
[0078] Word vector embedding uses the entire IDS dictionary as corpus to train a Word2vec model to obtain word embedding vectors for all radicals and structures; the average word embedding of the IDS sequence is taken as the embedding representation of each category:
[0079] (4)
[0080] in, Indicates radical or structure Word2Vec word embedding, L represents the length of IDS.
[0081] In step S3, the encoding network DenseNet169 first obtains a feature map through a convolution layer, and then performs a nonlinear transformation through a ReLU activation function; then, a series of dense blocks are used to gradually increase the dimension and complexity of the feature to obtain high-level abstract features of the image, remove the last classification layer, and take the flattened feature vector after average pooling as the real feature. Provides feature generation network learning.
[0082] In step S4, the input dimension of the feature generation network FGN is aligned with the corresponding semantic vector. The feature generation network FGN includes a feature generation VAE and a feature generation WGAN, which are denoted as f-VAE and f-WGAN. The f-VAE consists of an encoder Composition; f-WGAN consists of a generator and a discriminator The discriminator can distinguish the real features , and can also identify the generated features ; The f-VAE will utilize the generator in f-WGAN To assist in training f-VAE; the encoder The real features of the Chinese character image extracted by DenseNet169 are input Encoded into hidden state , and then by With real characteristics For the template Reconstruction to obtain generated features .
[0083] In step 4, the f-VAE is passed through a VAE loss function Optimize to enable the encoder to better parse Chinese character image features and obtain more representative hidden states Vector representation,
[0084] (5)
[0085] in, is the KL divergence, which is used to measure the potential spatial distribution of the encoder output With prior distribution The difference between , Indicates that the real feature and a semantic vector As an encoder Input and output hidden state ,in Used to indicate or or It means that in a given semantic vector Hidden state The prior distribution of It is the binary cross entropy used to calculate the real feature reconstruction loss, the hidden state of the encoder output The expected value of sampling, represent Relative to the encoder expectations, among which Indicates that the hidden state and semantic vector As a generator The input of the generator The output of is the generated feature ; The generated features synthesized by the generator are minimized by the VAE loss function As close to the real features as possible .
[0086] In step 4, the Wasserstein GAN loss is introduced To constrain the generator so that the generator Correctly embed the semantic vector Reconstruct back to the real features :
[0087] (6)
[0088] in, Representation Discriminator For real features and semantic vector Expected rating, Representation Discriminator Generate features and semantic vector Expected rating of represents the gradient penalty term, which is used to satisfy the 1-Lipschitz constraint. f-WGAN requires the discriminator is 1-Lipschitz continuous, which means that for any input, the norm of its gradient should be close to 1; where In real characteristics and generate features The features obtained by random interpolation between Indicates that the discriminator is and The second norm of the gradient on , is a penalty term used to make the norm of the gradient close to 1. is the penalty coefficient, which is used to control the weight of the gradient penalty term;
[0089] For the generator Introduced a classification loss To generate features that are more suitable for classification, such as Figure 5 The corresponding classifier is shown, and its loss function is:
[0090] (7)
[0091] in, Represents generated features The Chinese character category labels predicted to be true The probability of Represents the probability distribution The generated features obtained by sampling The expected value of the conditional probability is given by the parameter The fully connected layer of the classifier is pre-trained based on the actual features of the categories seen;
[0092] The overall optimization function of the feature generation network is:
[0093] (8)
[0094] (9)
[0095] in and Yes Control and The hyperparameter of the loss weight, formula (9) represents the objective function optimized by the feature generation network FGN, where , and Represent the generator, encoder and discriminator respectively, and the optimization strategy is to minimize and while maximizing the loss loss.
[0096] In step S6, the semantic description sequence IDS of each unknown Chinese character is known, and its three semantic vectors are obtained, namely, , , ; The three semantic vectors are combined with random Gaussian noise Then they are input into the corresponding trained feature generation network FGN respectively. The three feature generation networks FGN generate corresponding unseen class training feature samples according to the semantic embedding vectors. , and Repeat the above feature generation process, traverse each unseen Chinese character IDS to obtain a generated sample set that mixes the three semantics .
[0097] In step S7, the recognition model is optimized by the minimum negative log-likelihood loss:
[0098] (10)
[0099] in, Indicates that the real feature Map to The parameters of the fully connected layer of the Chinese character category are Optimize to minimize the objective function, that is , is the training dataset, Represents the true Chinese character category label calculated by softmax The probability distribution of is:
[0100] (11)
[0101] in, Is the real Chinese character category label The parameter vector and true feature The inner product of is the parameter vector and true feature of the mth Chinese character category The inner product of .
[0102] The recognition function of the recognition model trained in step S8 is:
[0103] (12)
[0104] in, Represents real characteristics, Represents the Chinese character category label.
[0105] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A zero-sample Chinese character recognition method based on diverse feature generation, characterized by: The steps include: Step S1: By expressing the radical and structure of each Chinese character into a tree structure, the tree structure is traversed in pre-order to obtain the semantic description sequence IDS of the Chinese character, which is expressed as , y represents the corresponding Chinese character category label; wherein, the ideographic description sequence is divided into a seen ideographic description sequence and an unseen ideographic description sequence according to whether it corresponds to a seen Chinese character image; Step S2: embed the semantic description sequence of each Chinese character from three directions: hierarchical decomposition, radical self-information, and word vector to obtain three corresponding semantic vectors , and ; Step S3: The seen Chinese character image is encoded into a feature map through the DenseNet169 network ,in , and Represent the height, width and number of channels of the feature map respectively, and then flatten it to get a The true characteristics of the dimension , and the true features of all the seen Chinese character images Constitute a real sample; Step S4: Three semantic vectors of Chinese characters , and Assign a feature generation network FGN to learn each semantic vector to the real feature The mapping of Step S5: Use the seen class samples to train the feature generation network FGN so that the FGN generates features Gradually approaching the true characteristics The distribution of the seen class samples includes the true features of all seen class Chinese character images , and the three semantic vectors of all the corresponding semantic description sequences , and ; Step S6: Generate unseen class training feature samples by using the three semantic vectors obtained by embedding each unseen class semantic description sequence through the trained three feature networks FGN , and , obtain the unseen class training feature samples of all unseen class semantic description sequences , and As a generated sample set ; Step S7: Generate sample set using and its corresponding Chinese character category labels, and the real samples and their corresponding Chinese character category labels together to train the recognition model so as to enable the recognition model to recognize Chinese character categories that have not been seen; Step S8: extract features of the image to be identified through the DenseNet169 network and then input it into the trained recognition model to obtain the result.
2. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 1, characterized in that: In the step S1, Chinese characters are decomposed into radicals and structural sequence representations to be used as common information between the seen Chinese characters and the unseen Chinese characters.
3. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 1, characterized in that: In step S2, the hierarchical decomposition embedding decomposes the Chinese character radical tree structure into For node representation, given a length of Node-path ,in, Represents the path connecting the nodes, and its influence value It is calculated by the following formula: (1) in, represents the influence coefficient that gradually decays as the node-path length increases, Indicates Path The influence coefficient of Indicates Path The influence coefficient of Indicates Path The influence value between the child nodes in the corresponding parent node; all nodes or node-paths in the radical tree are divided into two groups; one group is the leaf nodes representing the radical The set is represented by , and the other group is the parent node representing the structure The set is represented by ; Finally, the layer embedding vector It is expressed as: (2) in, and Respectively represent the radical and structure in IDS, is a hyperparameter that balances leaf nodes and parent nodes, Represents a radical node The impact value of Represents a structure node The impact value of is calculated by formula (1); Radical self-information embedding measures the key value of each radical from the perspective of information theory; then assign an index to each radical and structure and obtain its one-hot encoding, and obtain the radical self-information embedding vector through weighted encoding: (3) in, Indicated by IDS description sequence consisting of radicals and structures, Indicates the corresponding radical or structure after one-hot encoding, It is the probability distribution of radicals or structures, obtained by counting the IDS dictionary; Word vector embedding uses the entire IDS dictionary as corpus to train a Word2vec model to obtain word embedding vectors for all radicals and structures; the average word embedding of the IDS sequence is taken as the embedding representation of each category: (4) in, Indicates radical or structure Word2Vec word embedding, L represents the length of IDS.
4. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 1, characterized in that: In step S3, the encoding network DenseNet169 first obtains a feature map through a convolution layer, and then performs a nonlinear transformation through a ReLU activation function; then, a series of dense blocks are used to gradually increase the dimension and complexity of the feature to obtain high-level abstract features of the image, remove the last classification layer, and take the flattened feature vector after average pooling as the real feature. Provides feature generation network learning.
5. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 1, characterized in that: In step S4, the input dimension of the feature generation network FGN is aligned with the corresponding semantic vector. The feature generation network FGN includes a feature generation VAE and a feature generation WGAN, which are denoted as f-VAE and f-WGAN. The f-VAE consists of an encoder Composition; f-WGAN consists of a generator and a discriminator The discriminator can distinguish the real features , and can also identify the generated features ; The f-VAE will utilize the generator in f-WGAN To assist in training f-VAE; the encoder The real features of the Chinese character image extracted by DenseNet169 are input Encoded into hidden state , and then by With real characteristics For the template Reconstruction to obtain generated features .
6. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 5, characterized in that: The f-VAE uses a VAE loss function Optimize to enable the encoder to better parse Chinese character image features and obtain more representative hidden states Vector representation, (5) in, is the KL divergence, which is used to measure the potential spatial distribution of the encoder output With the prior distribution The difference between , Indicates that the real feature and a semantic vector As an encoder Input and output hidden state ,in Used to indicate or or It means that in a given semantic vector Hidden state The prior distribution of It is the binary cross entropy used to calculate the real feature reconstruction loss, the hidden state of the encoder output The expected value of sampling, represent Relative to the encoder expectations, among which Indicates that the hidden state and semantic vector As a generator The input of the generator The output of is the generated feature ; The generated features synthesized by the generator are minimized by the VAE loss function As close to the real features as possible .
7. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 6, characterized in that: Introducing Wasserstein GAN loss To constrain the generator so that the generator Correctly embed the semantic vector Reconstruct back to the real features : (6) in, Representation Discriminator For real features and semantic vector Expected rating, Representation Discriminator Generate features and semantic vector Expected rating of represents the gradient penalty term, which is used to satisfy the 1-Lipschitz constraint. In real characteristics and generate features The features obtained by random interpolation between Indicates that the discriminator is and The second norm of the gradient on , is a penalty term used to make the norm of the gradient close to 1. is the penalty coefficient, which is used to control the weight of the gradient penalty term; For the generator Introduced a classification loss To generate features more suitable for classification: (7) in, Represents generated features The Chinese character category labels predicted to be true The probability of Represents the probability distribution The generated features sampled from The expected value of the conditional probability is given by the parameter Fully connected layer calculation; The overall optimization function of the feature generation network is: (8) (9) in and Yes Control and The hyperparameter of the loss weight, formula (9) represents the objective function optimized by the feature generation network FGN, where , and Represent the generator, encoder and discriminator respectively, and the optimization strategy is to minimize and while maximizing the loss loss.
8. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 1, characterized in that: In step S6, the semantic description sequence IDS of each unknown Chinese character is known, and its three semantic vectors are obtained, namely, , , ; The three semantic vectors are combined with random Gaussian noise Then they are input into the corresponding trained feature generation network FGN respectively. The three feature generation networks FGN generate corresponding unseen class training feature samples according to the semantic embedding vectors. , and Repeat the above feature generation process, traverse each unseen Chinese character IDS to obtain a generated sample set that mixes the three semantics .
9. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 1, characterized in that: The recognition model in step S7 is optimized by the minimum negative log-likelihood loss: (10) in, Indicates that the real feature Map to The parameters of the fully connected layer of the Chinese character category are Optimize to minimize the objective function, that is , is the training dataset, Represents the true Chinese character category label calculated by softmax The probability distribution of is: (11) in, Is the real Chinese character category label The parameter vector and true feature The inner product of is the parameter vector and true feature of the mth Chinese character category The inner product of .
10. The zero-sample Chinese character recognition method based on diverse feature generation according to claim 9, characterized in that: The recognition function of the recognition model trained in step S8 is: (12) in, Represents real characteristics, Represents the Chinese character category label.
