A method for handwritten Chinese character recognition

Abstract

The handwritten Chinese character recognition method provided by the present invention comprises the following steps: Step S1, assigning labels to multiple Chinese characters, using multiple Chinese characters with the same label in the same domain as samples to train a network to obtain model parameters, randomly selecting examples from the samples, and computationally generating multiple prototypes, each of which represents a class in a deep feature space and is obtained by calculating the class center of each class; Step S2, memorizing the model parameters using soft knowledge distillation; Step S3, enhancing the prototypes; Step S4, using a gating mechanism to filter out the enhanced prototypes near the class center; Step S5, training the network using new samples, the filtered enhanced prototypes, and the examples, and repeating Steps S1 to S4. Compared to traditional joint training methods, the handwritten Chinese character recognition method of the present invention can achieve better results using less computing power and memory space.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a handwritten Chinese character recognition method. Background Art

[0002] Handwritten Chinese character recognition (HCCR) is an important research area in computer vision. Generally speaking, the goal of HCCR is to transcribe handwritten text into digital content. Unlike English and digit recognition, Chinese handwriting recognition is challenging, primarily due to the sheer size of the Chinese character categories and the rich variety of handwriting styles. This presents several challenges for the task. Currently, Chinese handwriting recognition faces the challenges of insufficient training data, primarily in terms of the number of handwriting categories and the number of handwriting styles. Existing recognition techniques fall into three main categories: character-based, radical-based, and stroke-based. Character-based methods use artificially designed features, such as directional and vector features, to identify characters. In 2015, Ciresan first applied CNNs (Convolutional Neural Networks) to image feature extraction for Chinese character recognition, proposing the MCDNN (Multi-column Deep Neural Network). Since then, several methods have been proposed using deep learning to address the problem of handwritten Chinese character recognition. However, character-based methods rely on large amounts of data and can only process existing characters, resulting in limitations. Radical-based methods such as FewShotRAN (Radical Aggregation Network) and HDE (Hierarchical Decomposition Embedding) can partially address the zero-shot HCCR problem, thereby alleviating the problem of insufficient handwriting data. Existing stroke-based methods mainly rely on traditional strategies that require manually designed features and are difficult to apply in practice.

[0003] However, most modern deep learning-based Chinese handwriting recognition networks address the problem of insufficient character categories through radical-based or stroke-based methods. However, in response to the problem of insufficient number of fonts, existing multi-author handwriting recognition methods usually assume that the number of fonts is limited. These methods mainly address the problem of insufficient character styles by training on existing data, hoping to extract most of the character styles in the existing data. Liu Chenglin et al. proposed a new adaptive layer method to reduce the mismatch between training and test data on a specific source layer. Despite limited success, it is clear that these networks tend to overfit the captured types or cause domain drift during domain adaptation, resulting in forgetting previously learned knowledge. In addition, considering that different people have different writing styles, there will always be new individuals with new writing styles. Therefore, in network training, it is necessary to consider how to improve recognition capabilities when there are more and more handwriting fonts. Summary of the Invention

[0004] Based on this, it is necessary to propose a handwritten Chinese character recognition method to improve the computer's recognition ability of handwritten Chinese characters.

[0005] The present invention provides a method for recognizing handwritten Chinese characters, comprising the following steps:

[0006] Step S1, assigning labels to multiple Chinese characters, using multiple Chinese characters with labels in the same domain as samples to train a network to obtain model parameters, randomly selecting examples from the samples, and generating multiple prototypes by calculation, wherein each prototype represents a class in a deep feature space, and each prototype is obtained by calculating the class center of each class;

[0007] Step S2, using soft knowledge distillation to memorize the model parameters;

[0008] Step S3, enhancing the prototype;

[0009] Step S4, using a gating mechanism to filter out the enhanced prototypes near the category center;

[0010] Step S5, using the new sample, the screened enhanced prototype and the example to train the network, and repeating steps S1 to S4.

[0011] Preferably, soft knowledge distillation adopts a reservoir sampling method to select

[0012] random samples, seeking to minimize the following objective:

[0013]

[0014] Among them, α is a hyperparameter used as a balancing factor, is the loss value from the beginning to the current task,

[0015] Add an item to the buffered data point, the final goal is:

[0016]

[0017] Where x' is the image stored in the reservoir, y' is the corresponding label, and β is the balancing factor.

[0018] Preferably, the prototype corresponding to each class is:

[0019]

[0020] where k is the class label, N t,k represents the labeled samples used for training in stage t, F θ Feature extractor.

[0021] Preferably, step S4 includes adding a gate γ to control the enhancement process:

[0022]

[0023] where γ represents the strict, standard, and relaxed states of the gate.

[0024] The handwritten Chinese character recognition method provided by the present invention comprises the following steps: Step S1, assigning labels to multiple Chinese characters, using multiple Chinese characters with the same label in the same domain as samples to train a network to obtain model parameters, randomly selecting examples from the samples, and generating multiple prototypes by calculation, wherein each prototype represents a class in a deep feature space, and each prototype is obtained by calculating the class center of each class; Step S2, using soft knowledge distillation to memorize the model parameters; Step S3, enhancing the prototypes; Step S4, using a gating mechanism to filter out the enhanced prototypes near the class center; Step S5, using new samples, the filtered enhanced prototypes, and the examples to train the network, and repeating Steps S1 to S4. Compared with traditional joint training methods, the handwritten Chinese character recognition method of the present invention can achieve better results using less computing power and memory space. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A flowchart of a handwritten Chinese character recognition method provided in one embodiment. DETAILED DESCRIPTION

[0027] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] As used in this application, the terms "component," "module," and "system" are intended to refer to a computer-related entity, which can be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable code, a thread of execution, a program, and / or a computer. As an illustration, both an application running on a server and a server can be a component. One or more components can reside in a process and / or a thread of execution, and a component can be located within a computer and / or distributed between two or more computers.

[0029] As used herein, the terms "inference" or "inference" generally refer to the process of inferring or reasoning about the state of a system, environment, and / or user from a set of observations captured via events and / or data. For example, inference can be used to identify a specific context or action, or can generate a probability distribution over states. Inference can be probabilistic, that is, a probability distribution over states of interest is computed based on a consideration of data and events. Inference can also refer to techniques for synthesizing higher-level events from a set of events and / or data. Such inference results in the construction of new events or actions from a set of observed events and / or stored event data, regardless of whether the events are related in adjacent time and regardless of whether the events and data come from one or several event and data sources.

[0030] like Figure 1As shown in the figure, taking the recognition of the Chinese character "spot" as an example, in step S1 of the handwritten Chinese character recognition method provided by the present invention, multiple Chinese characters are assigned labels, and the samples in the same domain using the labels are used to train the stage 0 network to obtain model parameters. A small part of the data is randomly selected from the multiple samples and stored as exemplars, and multiple prototypes are generated through calculation, where each of the prototypes represents a class in the deep feature space, and each of the prototypes is obtained by calculating the class center of each of the classes. In this way, in the new stage (stage 1), only the data, exemplars, and prototypes of stage 0 can be accessed by the training network. The training network not only extracts the knowledge learned in the old stage but also trains the accessible current stage data. During the training process, specific interference is added to each prototype, and by sampling samples near the prototype, the influence of outliers on the overall recognition performance can be excluded.

[0031] In step S2, soft knowledge distillation (soft knowledge distillation with replay, SKD) and experience replay (Dark Experience Replay, DER) are used to preserve the knowledge learned in the previous stage. Specifically, the model parameters of stage 0 are memorized using soft knowledge distillation.

[0032] DER represents the output logic corresponding probability distribution where x is the input picture.

[0033] DER adopts the reservoir sampling method to select random samples from the input stream. DER seeks to minimize the following objective:

[0034]

[0035] To handle sudden distribution shifts in the input stream, DER adds a term to the buffered data points, and the final objective is:

[0036]

[0037] In step S3, character prototype augmentation (CPA) is performed, that is, the prototypes generated through calculation in step S1 are enhanced to obtain enhanced prototypes.

[0038] To reduce the loss of information in the old domain feature space when training data in the new domain, a prototype is calculated and stored for each class:

[0039]

[0040] In the new phase, the prototype of each class in the old phase is expanded. old Class k old For example, the prototype is expanded like this:

[0041]

[0042] in is Gaussian noise. r is the scale used to control the uncertainty of the augmented prototype, which can be predefined or calculated as the average variance of the prototype.

[0043] Modify the calculation method of r to be defined as:

[0044]

[0045] Where K is the number of classes and R is the dimension of the feature space. t,k Represents the covariance matrix of class k features at stage t, and the Tr operation calculates the matrix Σ t,k In the above formula, r t is adaptively updated in the new stage by the data of each class, which means that when the domain shifts, r t Can be updated adaptively.

[0046] In step S4, a strict gate mechanism (SGM) is used to select reliable samples to represent the knowledge learned in the previous stage and reduce the influence of outliers. Specifically, the gate mechanism is used to filter out the enhanced prototypes near the center of the category.

[0047] Use the gating mechanism to keep unreliable samples out of the gate and let reliable samples in. Specifically, add a gate γ to control the enhancement process:

[0048]

[0049] Here, γ represents the three states of the gate: strict, standard, and relaxed, inspired by the three standard deviations of a Gaussian distribution. When the gate is in the strict state, only samples within one times the variance of the prototype can enter the experience replay phase in the new phase, meaning that 68.27% of samples can be resampled through the prototype enhancement process. Standard gata allows samples within the range of μ±2δ to be replayed. The relaxed gate allows samples within the range of μ±3δ to be resampled, meaning that it only excludes 0.26% of samples, which are outliers.

[0050] In step S5, the network is trained using the new sample, the screened enhanced prototype, and the example, and steps S1 to S4 are repeated. At this point, the network training enters a new stage (stage 1), and the new sample is the sample available for training in the new stage.

[0051] Ultimately, the overall goals of the overall assessment framework are as follows:

[0052]

[0053] Among them F i In the old stage, it is the class set C i Augmented features. It is clear that in the new stage, prototypes augmented with soft variance are fed into the network together with new data to maintain discrimination and achieve a balance between different domains.

[0054] Figure 1 We present RePA, a framework proposed for the StyleEL-HCCR problem. When only access to new-style data and a small portion of old-style data is allowed, RePA achieves strong performance on both new and old-style data. RePA consists of three modules: soft knowledge distillation (SKD), character prototype augmentation (CPA), and strict gating mechanism (SGM). SKD memorizes knowledge through knowledge distillation and replay, CPA learns representation information by memorizing character representation prototypes and augmenting them in new learning stages, and SGM improves reliability by using a strict gating mechanism.

[0055] When only 200 samples are stored, the method of the present invention is significantly better than other methods, which confirms that RePA can memorize more effective knowledge and better solve the problems of catastrophic forgetting and style increment. The present invention further explored the performance of the method of the present invention on a larger buffer and found that when the buffer size is 75100, which is 28.6% of the sample size, RePA performs better than joint training. This means that compared with traditional joint training methods, the method of the present invention can use less computing power and memory space to obtain better results. The reason why RePA is better than joint training is that the reliable prototype enhancement method enables the network to learn more effective handwritten Chinese character pattern knowledge.

[0056] The handwritten Chinese character recognition method provided by the present invention comprises the following steps: Step S1, assigning labels to multiple Chinese characters, using multiple Chinese characters with the same label in the same domain as samples to train a network to obtain model parameters, randomly selecting examples from the samples, and generating multiple prototypes through calculation, wherein each prototype represents a class in a deep feature space, and each prototype is obtained by calculating the class center of each class; Step S2, using soft knowledge distillation to memorize the model parameters; Step S3, enhancing the prototypes; Step S4, using a gating mechanism to filter out the enhanced prototypes near the class center; Step S5, using new samples, the filtered enhanced prototypes, and the examples to train the network, and repeating Steps S1 to S4. Compared with traditional joint training methods, the handwritten Chinese character recognition method of the present invention can achieve better results using less computing power and memory space.

[0057] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A handwritten Chinese character recognition method, characterized in that: The following steps are involved: Step S1, assigning labels to multiple Chinese characters, using multiple Chinese characters with labels in the same domain as samples to train a network to obtain model parameters, randomly selecting examples from the samples, and generating multiple prototypes by calculation, wherein each prototype represents a class in a deep feature space, and each prototype is obtained by calculating the class center of each class; Step S2, using soft knowledge distillation to memorize the model parameters; Step S3, enhancing the prototype; Step S4, using a gating mechanism to filter out the enhanced prototypes near the category center; Step S5, using the new sample, the screened enhanced prototype and the example to train the network, repeating steps S1 to S4, The soft knowledge distillation adopts the reservoir sampling method to select random samples, seeking to minimize the following objective: Among them, α is a hyperparameter used as a balancing factor, is the loss value from the beginning to the current training stage, Add an item to the buffered data point, the final goal is: Where x' is the image stored in the reservoir, y' is the corresponding label, and β is the balancing factor.

2. The handwritten Chinese character recognition method according to claim 1, wherein: The prototype corresponding to each class is: where k is the class label, N t,k represents the labeled samples used for training in stage t, F θ Feature extractor.

3. The handwritten Chinese character recognition method according to claim 1, wherein: The step S4 includes adding a gate γ to control the enhancement process: where γ represents the strict, standard, and relaxed states of the gate.

Images