Handwriting recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a handwriting recognition method and device, electronic equipment and storage medium, comprising: obtaining training data of simulated free writing, sampling and processing stroke coordinate points to obtain sampling data, extracting features of the sampling data through a preset residual convolutional network, generating high-level features corresponding to the sampling data, and collaboratively training the preset residual convolutional network in an end-to-end manner according to a preset character segmentation network and a preset CTC character recognition network, and analyzing the high-level features through the preset CTC character recognition network during prediction to obtain a target recognition result. The present application takes the stroke coordinate points of free writing data as the processing unit for sampling and feature extraction, avoids the problem of inaccurate recognition caused by segmentation errors in single character segmentation and recognition, uses a shared one-dimensional residual convolutional neural network for feature extraction, and collaboratively trains the character segmentation network and the CTC network in an end-to-end manner, without the need for manual feature design, to achieve accurate handwriting input recognition and improve user input experience.

Description

Technical Field

[0001] This invention belongs to the field of handwriting input technology, and in particular relates to a handwriting recognition method, device, electronic device and storage medium. Background Technology

[0002] With the rapid development of Internet technology, various input methods are usually provided to facilitate user input. Among them, handwriting input is a very common input method on terminal devices, such as in-vehicle input and mobile phone input.

[0003] Currently, handwriting input recognition only supports overlapping writing or line writing. The reason for supporting only line writing or overlapping writing is that the underlying algorithms use different processing procedures. For example, overlapping writing recognition requires a character hypothesis model to perform accurate single-character segmentation and recognition, and path search decoding based on single-character recognition scores and language models. Line writing recognition requires the use of single-character segmentation or line recognition neural network algorithms. In essence, all of the above recognition methods use single-character segmentation and recognition.

[0004] However, single-character segmentation and recognition based on the continuous character segmentation assumption model outputs character segmentation points. If the character segmentation point is not accurately found, the character cannot be accurately recognized. If there are too many segmentation points, it will increase the complexity of recognition and path decoding. The probability of incorrect character segmentation is very high, making it easy to make segmentation errors. Furthermore, when characters form sentences, path search optimization is difficult, resulting in low recognition accuracy. Therefore, it does not support relatively arbitrary freehand input, affecting the user input experience. Summary of the Invention

[0005] This invention provides a handwriting recognition method, device, electronic device, and storage medium to solve the problems of low recognition accuracy and lack of support for free writing in existing handwriting input recognition methods.

[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0007] In a first aspect, the present invention provides a handwriting recognition method, the method comprising:

[0008] The training data for simulated free writing is obtained, and the stroke coordinate points are sampled to obtain sampled data, wherein the training data includes stroke coordinate points;

[0009] The sampling data is feature extracted by a preset residual convolutional network to generate high-level features corresponding to the sampling data.

[0010] Based on the preset character segmentation network and the preset CTC character recognition network, the preset residual convolutional network is trained end-to-end. During prediction, the high-level features are analyzed through the preset CTC character recognition network to obtain the target recognition result.

[0011] Optionally, the step of obtaining training data for simulated free writing involves sampling the stroke coordinate points to obtain sampled data. The training data includes stroke coordinate points, comprising:

[0012] Acquire training data for simulated free writing, wherein the training data is generated by a preset free writing simulation algorithm and includes at least line writing, overlapping writing, and mixed line-overlapping writing.

[0013] The training data is preprocessed using a cubic spline interpolation algorithm. If the stroke error is greater than a preset error range, the stroke is split into multiple strokes.

[0014] The stroke coordinates of multiple strokes are normalized.

[0015] Normalized stroke coordinate points are sampled at equal intervals to obtain sampled data.

[0016] Optionally, the step of extracting features from the sampled data using a preset residual convolutional network to generate high-level features corresponding to the sampled data includes:

[0017] By using a preset residual convolutional network, feature information is obtained by extracting features from the sampled data according to a preset feature dimension. The preset residual convolutional network includes multiple residual modules composed of convolutional layers, batch normalization layers, activation layers, and residual connection layers.

[0018] The feature information is input into the preset residual convolutional network, and the feature information is transformed to generate high-level features corresponding to the sampled data.

[0019] Optionally, the step of end-to-end collaboratively training the preset residual convolutional network based on a preset character segmentation network and a preset CTC character recognition network, and parsing the high-level features through the preset CTC character recognition network during prediction to obtain the target recognition result, includes:

[0020] Add a preset character segmentation network and a preset CTC network to the high-level features;

[0021] Based on the preset character segmentation network and the preset CTC character recognition network, the preset residual convolutional network is trained end-to-end in a collaborative manner.

[0022] During prediction, the high-level features are parsed using the preset CTC character recognition network;

[0023] Based on the loss function of the preset character segmentation network, the parsed high-level features are adjusted to obtain the target recognition result corresponding to the training data.

[0024] Optionally, adjusting the parsed high-level features based on the loss function of the preset character segmentation network to obtain the target recognition result corresponding to the training data includes:

[0025] Obtain the high-level features corresponding to the parsed training data;

[0026] Based on the loss function, character loss is obtained according to the training data and the parsed high-level features;

[0027] Based on the character loss, the high-level features are adjusted to obtain the target recognition result corresponding to the training data.

[0028] In a second aspect, the present invention provides a handwriting recognition device, the device comprising:

[0029] The data processing module is used to acquire training data for simulated free writing, and to sample the stroke coordinate points to obtain sampled data, wherein the training data includes stroke coordinate points.

[0030] The feature extraction module is used to extract features from the sampled data through a preset residual convolutional network to generate high-level features corresponding to the sampled data.

[0031] The training and recognition module is used to train the preset residual convolutional network end-to-end based on the preset character segmentation network and the preset CTC character recognition network. During prediction, the high-level features are parsed through the preset CTC character recognition network to obtain the target recognition result.

[0032] Optionally, the data processing module includes:

[0033] The first acquisition submodule is used to acquire training data for simulation free writing, wherein the training data is generated by a preset free writing simulation algorithm and includes at least line writing, overlapping writing, and mixed line-overlapping writing.

[0034] The preprocessing submodule is used to preprocess various training data using a cubic spline interpolation algorithm. If the stroke error is greater than a preset error range, the stroke is split into multiple strokes.

[0035] The first processing submodule is used to normalize the stroke coordinate points of the multiple strokes.

[0036] The second processing submodule is used to perform equidistant sampling of the normalized stroke coordinate points to obtain sampled data.

[0037] Optionally, the feature extraction module includes:

[0038] The first extraction submodule is used to extract features from the sampled data according to a preset feature dimension through a preset residual convolutional network to obtain feature information. The preset residual convolutional network includes multiple residual modules composed of convolutional layers, batch normalization layers, activation layers, and residual connection layers.

[0039] The first generation submodule is used to input the feature information into the preset residual convolutional network, transform the feature information, and generate high-level features corresponding to the sampled data.

[0040] Optionally, the training and recognition module includes:

[0041] Add a network submodule to add a preset character segmentation network and a preset CTC network on the high-level features;

[0042] The training submodule is used to train the preset residual convolutional network end-to-end in conjunction with the preset character segmentation network and the preset CTC character recognition network.

[0043] The parsing submodule is used to parse the high-level features through the preset CTC character recognition network during prediction;

[0044] The second acquisition submodule is used to adjust the parsed high-level features based on the loss function of the preset character segmentation network to obtain the target recognition result corresponding to the training data.

[0045] Optionally, the second acquisition submodule includes:

[0046] The acquisition unit is used to acquire the high-level features corresponding to the parsed training data;

[0047] A loss unit is used to obtain character loss based on the loss function, the training data, and the parsed high-level features;

[0048] The generation unit is used to adjust the high-level features according to the character loss to obtain the target recognition result corresponding to the training data.

[0049] Thirdly, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described handwriting recognition method.

[0050] Fourthly, the present invention provides a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described handwriting recognition method.

[0051] This invention, in its embodiments, acquires training data from simulated freehand writing, samples the stroke coordinates to obtain sampled data, extracts features from the sampled data using a pre-defined residual convolutional network, generates high-level features corresponding to the sampled data, and trains the pre-defined residual convolutional network end-to-end based on a pre-defined character segmentation network and a pre-defined CTC character recognition network. During prediction, the high-level features are analyzed by the pre-defined CTC character recognition network to obtain the target recognition result. This invention uses the stroke coordinates of simulated freehand writing data as the processing unit for sampling and feature extraction, avoiding the problem of inaccurate recognition due to segmentation errors in single-character segmentation and recognition algorithms. It utilizes a shared one-dimensional residual convolutional neural network for feature extraction, and performs end-to-end training through the collaborative use of the character segmentation network and the CTC network. This eliminates the need for manual feature design, achieving accurate recognition of handwritten input and improving the user input experience. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the steps of a handwriting recognition method provided in an embodiment of the present invention;

[0054] Figure 2 yes Figure 1 The present invention provides a flowchart of step 101 of a handwriting recognition method;

[0055] Figure 3 yes Figure 1 A flowchart of step 102 of a handwriting recognition method provided in this embodiment of the invention;

[0056] Figure 4 yes Figure 1 A flowchart of step 103 of a handwriting recognition method provided in this embodiment of the invention;

[0057] Figure 5 This is a schematic diagram of the structure of a handwriting recognition device provided in an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Reference Figure 1 The flowchart illustrates the steps of a handwriting recognition method provided in an embodiment of the present invention, the method including:

[0061] Step 101: Obtain training data for simulated free writing by sampling the stroke coordinate points to obtain sampled data, where the training data includes stroke coordinate points.

[0062] In this embodiment of the invention, in order to recognize the freehand writing in the input method uploaded by the user, it is necessary to obtain a suitable online freehand writing simulation algorithm to obtain training data generated by the simulation freehand writing. In this embodiment, open-source GBK (Chinese Internal Code Specification) online single-character characters can be collected, and the collected single-character characters can be simulated by line writing, overlapping writing, and mixed line and overlapping writing data to obtain training data generated by the simulation freehand writing.

[0063] It should be noted that free writing includes modes such as line writing, overlapping writing, and mixed line and overlapping writing. In the embodiments of the present invention, the training data generated by simulating free writing can also be obtained by pre-collecting free writing characters input by the user. The present invention does not make specific limitations here.

[0064] Specifically, in this embodiment, based on the training data generated by the simulated free writing, the stroke coordinate points of the training data are sampled using a cubic spline interpolation algorithm to obtain sampled data.

[0065] For example, the coordinate points of the input training data are divided into strokes, and the stroke coordinate points are normalized. Normalization is used to make the strokes uniform and standardized, which facilitates subsequent feature extraction. In this embodiment, the height can be normalized to 0-1, and the width can be normalized proportionally. The normalized strokes are sampled using a cubic spline interpolation algorithm. 20 points are sampled at equal intervals for each stroke. At this time, the output of a sample is T*20*2, and the dimension is transformed to T*40. T is the total number of strokes in this sample. For example, taking the simulated free writing training sample "one" as an example, the number of strokes is 1+3=4. After cubic spline interpolation, the output is 4*40 dimension sampled data to facilitate the next step of feature extraction.

[0066] In this embodiment, training data generated by simulated free writing is obtained, and the stroke coordinates of the training data are sampled using a cubic spline interpolation algorithm to obtain sampled data. The stroke coordinates of the simulated free writing data are used as the processing unit for sampling, which avoids the problem of inaccurate recognition due to the large granularity of single-character segmentation and recognition.

[0067] Step 102: Extract features from the sampled data using a preset residual convolutional network to generate high-level features corresponding to the sampled data.

[0068] It should be noted that the preset residual convolutional network can be a one-dimensional convolutional neural network used to extract stroke features. In this embodiment, the residual convolutional network can effectively alleviate the gradient vanishing problem, greatly improve the depth of the effective training network, and thus improve the accuracy of handwriting recognition.

[0069] Specifically, a preset residual convolutional network is used to extract features from the sampled data to generate high-level features corresponding to the sampled data. For example, a residual conv1d network is used to extract features from the sampled data of multiple training samples in a batch, and the output dimension is B*T*D, where B is the batch size, T is the data with the maximum number of strokes in the batch, and D is the feature dimension of a single stroke extracted by the residual conv1d network. The fully connected layer in the preset residual convolutional network is used to transform the features to generate high-level features corresponding to the sampled data.

[0070] Step 103: Based on the preset character segmentation network and the preset CTC character recognition network, the preset residual convolutional network is trained end-to-end. During prediction, the high-level features are analyzed through the preset CTC character recognition network to obtain the target recognition result.

[0071] It should be noted that the preset character segmentation network and the preset CTC network are pre-trained. The preset character segmentation network includes several residual networks, convolutional layers, fully connected layers, and a loss function. An attention mechanism is set after several residual networks to make the character segmentation model pay more attention to meaningful and important channels to improve character segmentation accuracy. The CTC network (Connectionist temporal classification) is a text recognition algorithm network used to solve the problem of inconsistent lengths and misalignment between user input and output sequences. The shared one-dimensional residual convolutional network contains multiple one-dimensional residual modules. Each residual module contains a one-dimensional convolutional layer, a one-dimensional batch normalization layer, an activation layer, and a residual connection layer. In this embodiment, the one-dimensional residual convolutional network is trained end-to-end based on the preset character segmentation network and the preset CTC character recognition network to generate the target recognition result corresponding to the training data based on the high-level features generated by the one-dimensional residual convolutional network.

[0072] Specifically, a preset character segmentation network and a preset CTC network are added to the high-level features. Based on the preset character segmentation network branch and the preset CTC character recognition network branch, an end-to-end collaborative training optimizes the shared one-dimensional residual convolutional network. During prediction, the high-level features are parsed through the preset CTC character recognition network to obtain the target recognition result. First, during prediction, the high-level features are parsed through the preset CTC character recognition network. Based on the loss function of the preset character segmentation network, the high-level features are adjusted to obtain the target recognition result corresponding to the training data.

[0073] This invention, in its embodiments, acquires training data from simulated freehand writing, samples the stroke coordinates to obtain sampled data, extracts features from the sampled data using a pre-defined residual convolutional network, generates high-level features corresponding to the sampled data, and trains the pre-defined residual convolutional network end-to-end based on a pre-defined character segmentation network and a pre-defined CTC character recognition network. During prediction, the high-level features are analyzed by the pre-defined CTC character recognition network to obtain the target recognition result. This invention uses the stroke coordinates of simulated freehand writing data as the processing unit for sampling and feature extraction, avoiding the problem of inaccurate recognition due to segmentation errors in single-character segmentation and recognition algorithms. It utilizes a shared one-dimensional residual convolutional neural network for feature extraction, and performs end-to-end training through the collaborative use of the character segmentation network and the CTC network. This eliminates the need for manual feature design, achieving accurate recognition of handwritten input and improving the user input experience.

[0074] Furthermore, refer to Figure 2 , Figure 2 yes Figure 1 The flowchart of step 101 of the handwriting recognition method provided in this embodiment of the invention includes:

[0075] Step 1011: Obtain training data for simulated free writing, wherein the training data is generated by a preset free writing simulation algorithm and includes at least line writing, overlapping writing, and mixed line-overlapping writing.

[0076] Step 1012: Preprocess various training data using cubic spline interpolation algorithm. If the stroke error is greater than the preset error range, the stroke is split into multiple strokes.

[0077] Specifically, in this embodiment of the invention, various training data are preprocessed using a cubic spline interpolation algorithm. If the error after interpolation preprocessing is large, such as the presence of connected strokes, the stroke needs to be split into multiple strokes at the inflection point. Therefore, if the stroke error is greater than a preset error range, the stroke is split into multiple strokes. It should be noted that the preset error range is pre-set based on the actual stroke division and is not specifically limited here.

[0078] Step 1013: Normalize the stroke coordinate points of a plurality of strokes.

[0079] In this embodiment, a residual convolutional network is used. Since the input layer of the residual convolutional network can process multidimensional data, in order to facilitate processing training data with stroke coordinate points as the processing unit and further perform feature extraction through the residual convolutional network, this embodiment needs to perform normalization processing on the training data before sampling. For example, the input layer of a one-dimensional convolutional network receives a one-dimensional or two-dimensional array, where the one-dimensional array is usually time or spectrum sampling; the input layer of a two-dimensional convolutional neural network receives a two-dimensional or three-dimensional array; the input layer of a three-dimensional convolutional neural network receives a four-dimensional array.

[0080] This embodiment does not specifically limit the normalization processing of stroke coordinate points. The purpose of the normalization processing is to represent each stroke with an image of a fixed size, so as to facilitate feature extraction through the residual convolutional network. In the actual use process, the stroke coordinate points can be normalized to any standard that can uniquely indicate the stroke.

[0081] Step 1014: Perform equidistant sampling processing on the normalized stroke coordinate points to obtain sampled data.

[0082] For example, the normalized stroke coordinate points are fitted by a cubic spline interpolation algorithm, that is, equidistant sampling is performed on the normalized stroke coordinate points using cubic spline interpolation, and the output is T*N*2, where T is the number of strokes, N is the number of sampling points per stroke, and 2 is the dimension of x and y coordinates, and the sampled data of T*2N is obtained after dimension transformation.

[0083] It should be noted that each stroke is composed of a combination of point sets. For example, the stroke "|" may be composed of 30 point sets. In order to separate the strokes, it is necessary to first obtain the stroke coordinate points constituting the character string. In this embodiment, equidistant sampling is performed on the normalized stroke coordinate points through a cubic spline interpolation algorithm. Of course, the above is only a specific illustrative example. In the actual use process, the sampling distance, that is, the number of sampling points, is preset according to actual recognition requirements, and sampling may be performed at intervals of 2 points or 3 points, which is not specifically limited in this embodiment.

[0084] Compared with the prior art, the embodiments of the present invention obtain training data generated by simulating free writing through a preset free writing simulation algorithm, perform normalization processing on the stroke coordinate points of the training data, and perform equidistant sampling processing on the normalized stroke coordinate points through a cubic spline interpolation algorithm to obtain sampled data, which can support training data of different types of free writing inputs, making the technical solution provided by the embodiments of the present invention more applicable.

[0085] Specifically, refer to Figure 3 , Figure 3 is Figure 1 The flowchart of step 102 of the handwriting recognition method provided in this embodiment of the invention includes:

[0086] Step 1021: Using a preset residual convolutional network, feature extraction is performed on the sampled data according to the preset feature dimensions to obtain feature information;

[0087] The preset residual convolutional network includes multiple residual modules consisting of convolutional layers, batch normalization layers, activation layers, and residual connection layers.

[0088] Step 1022: Input the feature information into a preset residual convolutional network, transform the feature information, and generate high-level features corresponding to the sampled data.

[0089] It should be noted that in steps 1021 to 1022 above, the preset residual convolutional network is pre-trained. The shared one-dimensional residual convolutional network contains multiple one-dimensional residual modules. Each residual module contains a one-dimensional convolutional layer, a one-dimensional batch normalization layer, an activation layer, and a residual connection layer. This embodiment will not elaborate on the residual convolutional network in detail. The preset residual convolutional network includes a residual connection layer, which is used for feature transformation. The residual convolutional network extracts features from the sampled data, and the output dimension is B*T*D, where B is the batch size, T is the data with the largest number of strokes in the batch, the number of strokes of other data in the same batch is flattened, and D is the feature dimension of a single stroke extracted by the residual convolutional network. The feature information is input to the first fully connected layer of the preset residual convolutional network to transform the feature information and generate high-level features corresponding to the sampled data. The high-level features corresponding to the sampled data facilitate the subsequent addition of a character segmentation network and a CTC network for final end-to-end training.

[0090] In this embodiment of the invention, a preset residual convolutional network is used to extract features from the sampled data according to a preset feature dimension to obtain feature information. The feature information is then transformed to generate high-level features corresponding to the sampled data. That is, feature extraction is performed using a residual convolutional neural network, eliminating the need for manual feature design. This enables accurate recognition of handwritten input and improves the user input experience.

[0091] Furthermore, refer to Figure 4 , Figure 4 yes Figure 1 The flowchart of step 103 of the handwriting recognition method provided in this embodiment of the invention includes:

[0092] Step 1031: Add a preset character segmentation network and a preset CTC network to the high-level features.

[0093] Specifically, a preset character segmentation network and a preset CTC network are added on top of the high-level features for final end-to-end training. The character segmentation network includes a fully connected layer and a loss function. The fully connected layer has 1 hidden unit, transforming the high-level features B*T*C into B*T*1. Then, the loss function is used to supervise character segmentation. The character segmentation network acts as an auxiliary network, working together with the CTC network to achieve better, faster, and more effective convergence, and improve the recognition rate.

[0094] Step 1032: Train the preset residual convolutional network end-to-end in collaboration with the preset character segmentation network and the preset CTC character recognition network.

[0095] Step 1033: During prediction, high-level features are analyzed using a pre-set CTC character recognition network.

[0096] In this embodiment, when recognizing and predicting freely written data, a preset CTC character recognition network is used to parse high-level features. The high-level features are then adjusted based on the prediction results to obtain accurate recognition results. The preset character segmentation network includes a loss function, which can be a sigmoid loss function. This function adds the feature vectors output by different networks to obtain a fused feature vector, thereby adjusting the high-level features and generating the target recognition result corresponding to the training data.

[0097] Step 1034: Based on the loss function of the preset character segmentation network, adjust the parsed high-level features to obtain the target recognition result corresponding to the training data.

[0098] Specifically, step 1034, based on the loss function of the preset character segmentation network, adjusts the parsed high-level features to obtain the target recognition result corresponding to the training data, which may include the following steps:

[0099] First, obtain the high-level features corresponding to the parsed training data;

[0100] Secondly, based on the loss function, character loss is obtained according to the training data and the parsed high-level features;

[0101] Secondly, based on the character loss, the high-level features are adjusted to obtain the target recognition results corresponding to the training data.

[0102] It should be noted that, firstly, the high-level features corresponding to the training data are obtained. The recognition results corresponding to the high-level features may have character loss. Therefore, based on the loss function of the preset character segmentation network, the character loss is obtained according to the training data and the parsed high-level features. Then, by calling the loss function of the character segmentation network, the high-level features are adjusted, and the target recognition results corresponding to the training data are regenerated to further improve the recognition accuracy.

[0103] The handwriting recognition method provided in this embodiment uses the stroke coordinates of simulated freehand data as the processing unit for sampling and feature extraction. This avoids the problem of inaccurate recognition due to the large granularity of single-character segmentation and recognition. Furthermore, it supports freehand input data without considering a single handwriting input method. It utilizes a residual convolutional neural network for feature extraction and employs a character segmentation network and a CTC network for end-to-end training. This eliminates the need for manual feature design, achieving accurate recognition of handwriting input and improving the user input experience.

[0104] Reference Figure 5 The diagram illustrates the structure of a handwriting recognition device provided in an embodiment of the present invention. The device may include:

[0105] The data processing module 201 is used to acquire training data for simulated free writing, and to sample the stroke coordinate points to obtain sampled data, wherein the training data includes stroke coordinate points;

[0106] Feature extraction module 202 is used to extract features from the sampled data through a preset residual convolutional network to generate high-level features corresponding to the sampled data;

[0107] The training and recognition module 203 is used to train the preset residual convolutional network end-to-end based on the preset character segmentation network and the preset CTC character recognition network. During prediction, the high-level features are parsed through the preset CTC character recognition network to obtain the target recognition result.

[0108] Furthermore, the data processing module 201 includes:

[0109] The first acquisition submodule is used to acquire training data for simulation free writing, wherein the training data is generated by a preset free writing simulation algorithm and includes at least line writing, overlapping writing, and mixed line-overlapping writing.

[0110] The preprocessing submodule is used to preprocess various training data using a cubic spline interpolation algorithm. If the stroke error is greater than a preset error range, the stroke is split into multiple strokes.

[0111] The first processing submodule is used to normalize the stroke coordinate points of the multiple strokes.

[0112] The second processing submodule is used to perform equidistant sampling of the normalized stroke coordinate points to obtain sampled data.

[0113] Furthermore, the feature extraction module 202 includes:

[0114] The first extraction submodule is used to extract features from the sampled data according to a preset feature dimension through a preset residual convolutional network to obtain feature information. The preset residual convolutional network includes multiple residual modules composed of convolutional layers, batch normalization layers, activation layers, and residual connection layers.

[0115] The first generation submodule is used to input the feature information into the preset residual convolutional network, transform the feature information, and generate high-level features corresponding to the sampled data.

[0116] Furthermore, the training recognition module 203 includes:

[0117] Add a network submodule to add a preset character segmentation network and a preset CTC network on the high-level features;

[0118] The training submodule is used to train the preset residual convolutional network end-to-end in conjunction with the preset character segmentation network and the preset CTC character recognition network.

[0119] The parsing submodule is used to parse the high-level features through the preset CTC character recognition network during prediction;

[0120] The second acquisition submodule is used to adjust the parsed high-level features based on the loss function of the preset character segmentation network to obtain the target recognition result corresponding to the training data.

[0121] Furthermore, the second acquisition submodule includes:

[0122] The acquisition unit is used to acquire the high-level features corresponding to the parsed training data;

[0123] A loss unit is used to obtain character loss based on the loss function, the training data, and the parsed high-level features;

[0124] The generation unit is used to adjust the high-level features according to the character loss to obtain the target recognition result corresponding to the training data.

[0125] This invention, in its embodiments, acquires training data from simulated freehand writing, samples the stroke coordinates to obtain sampled data, extracts features from the sampled data using a pre-defined residual convolutional network, generates high-level features corresponding to the sampled data, and trains the pre-defined residual convolutional network end-to-end based on a pre-defined character segmentation network and a pre-defined CTC character recognition network. During prediction, the high-level features are analyzed by the pre-defined CTC character recognition network to obtain the target recognition result. This invention uses the stroke coordinates of simulated freehand writing data as the processing unit for sampling and feature extraction, avoiding the problem of inaccurate recognition due to segmentation errors in single-character segmentation and recognition algorithms. It utilizes a shared one-dimensional residual convolutional neural network for feature extraction, and performs end-to-end training through the collaborative use of the character segmentation network and the CTC network. This eliminates the need for manual feature design, achieving accurate recognition of handwritten input and improving the user input experience.

[0126] This invention also provides an electronic device, such as... Figure 6 As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304.

[0127] Memory 303 is used to store computer programs;

[0128] When processor 301 executes a program stored in memory 303, it can perform the following steps:

[0129] Obtain training data for simulated free writing, and sample the stroke coordinate points to obtain sampled data;

[0130] Feature extraction is performed on the sampled data using a pre-defined residual convolutional network to generate high-level features corresponding to the sampled data;

[0131] Based on the preset character segmentation network and the preset CTC character recognition network, the preset residual convolutional network is trained end-to-end. During prediction, the preset CTC character recognition network is used to analyze high-level features to obtain the target recognition result.

[0132] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0133] The communication interface is used for communication between the aforementioned terminal and other devices.

[0134] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0135] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0136] The present invention also provides a readable storage medium, wherein when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the handwriting recognition method of the foregoing embodiments.

[0137] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0138] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. The structure required to construct such a system is readily apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0139] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0140] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0141] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0142] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0143] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0144] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0145] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention. However, the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should also be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0146] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

Claims

1. A handwriting recognition method, characterized in that, The method includes: The training data for simulated free writing is obtained, and the stroke coordinate points are sampled to obtain sampled data, wherein the training data includes stroke coordinate points; Feature extraction is performed on the sampled data using a preset residual convolutional network to generate high-level features corresponding to the sampled data; the preset residual convolutional network is a one-dimensional convolutional neural network. Based on the preset character segmentation network and the preset CTC character recognition network, the preset residual convolutional network is trained end-to-end. During prediction, the high-level features are parsed through the preset CTC character recognition network to obtain the target recognition result. The step of end-to-end collaboratively training the preset residual convolutional network based on a preset character segmentation network and a preset CTC character recognition network, and parsing the high-level features through the preset CTC character recognition network during prediction to obtain the target recognition result, includes: adding a preset character segmentation network and a preset CTC character recognition network to the high-level features; end-to-end collaboratively training the preset residual convolutional network based on the preset character segmentation network and the preset CTC character recognition network; parsing the high-level features through the preset CTC character recognition network during prediction; and adjusting the parsed high-level features corresponding to the training data based on the loss function of the preset character segmentation network to obtain the target recognition result corresponding to the training data; wherein the loss function is used to add the feature vectors output by different networks to obtain a fused feature vector, thereby adjusting the high-level features.

2. The method according to claim 1, characterized in that, The process of acquiring training data for simulated freehand writing involves sampling the stroke coordinate points to obtain sampled data. The training data includes stroke coordinate points, comprising: Acquire training data for simulated free writing, wherein the training data is generated by a preset free writing simulation algorithm and includes at least line writing, overlapping writing, and mixed line-overlapping writing. The training data is preprocessed using a cubic spline interpolation algorithm. If the stroke error is greater than a preset error range, the stroke is split into multiple strokes. The stroke coordinates of multiple strokes are normalized. Normalized stroke coordinate points are sampled at equal intervals to obtain sampled data.

3. The method according to claim 1, characterized in that, The step of extracting features from the sampled data using a preset residual convolutional network to generate high-level features corresponding to the sampled data includes: By using a preset residual convolutional network, feature information is obtained by extracting features from the sampled data according to a preset feature dimension. The preset residual convolutional network includes multiple residual modules composed of convolutional layers, batch normalization layers, activation layers, and residual connection layers. The feature information is input into the preset residual convolutional network, and the feature information is transformed to generate high-level features corresponding to the sampled data.

4. The method according to claim 1, characterized in that, The step of adjusting the parsed high-level features corresponding to the training data based on the loss function of the preset character segmentation network to obtain the target recognition result corresponding to the training data includes: Obtain the high-level features corresponding to the parsed training data; Based on the loss function, character loss is obtained according to the training data and the parsed high-level features; Based on the character loss, the high-level features are adjusted to obtain the target recognition result corresponding to the training data.

5. A handwriting recognition device, characterized in that, The device includes: The data processing module is used to acquire training data for simulated free writing, and to sample the stroke coordinate points to obtain sampled data, wherein the training data includes stroke coordinate points. The feature extraction module is used to extract features from the sampled data through a preset residual convolutional network to generate high-level features corresponding to the sampled data; the preset residual convolutional network is a one-dimensional convolutional neural network. The training and recognition module is used to train the preset residual convolutional network end-to-end based on the preset character segmentation network and the preset CTC character recognition network. During prediction, the high-level features are parsed through the preset CTC character recognition network to obtain the target recognition result. The training and recognition module includes: a network addition submodule, used to add the preset character segmentation network and the preset CTC character recognition network to the high-level features; a training submodule, used to collaboratively train the preset residual convolutional network end-to-end based on the preset character segmentation network and the preset CTC character recognition network; a parsing submodule, used to parse the high-level features through the preset CTC character recognition network during prediction; and a second acquisition submodule, used to adjust the parsed high-level features corresponding to the training data based on the loss function of the preset character segmentation network, and acquire the target recognition result corresponding to the training data; wherein, the loss function is used to add the feature vectors output by different networks to obtain a fused feature vector, thereby adjusting the high-level features.

6. The apparatus according to claim 5, characterized in that, The data processing module includes: The first acquisition submodule is used to acquire training data for simulation free writing, wherein the training data is generated by a preset free writing simulation algorithm and includes at least line writing, overlapping writing, and mixed line-overlapping writing. The preprocessing submodule is used to preprocess various training data using a cubic spline interpolation algorithm. If the stroke error is greater than a preset error range, the stroke is split into multiple strokes. The first processing submodule is used to normalize the stroke coordinate points of the multiple strokes. The second processing submodule is used to perform equidistant sampling of the normalized stroke coordinate points to obtain sampled data.

7. The apparatus according to claim 5, characterized in that, The feature extraction module includes: The first extraction submodule is used to extract features from the sampled data according to a preset feature dimension through a preset residual convolutional network to obtain feature information. The preset residual convolutional network includes multiple residual modules composed of convolutional layers, batch normalization layers, activation layers, and residual connection layers. The first generation submodule is used to input the feature information into the preset residual convolutional network, transform the feature information, and generate high-level features corresponding to the sampled data.

8. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the program, implements the handwriting recognition method as described in any one of claims 1-4.

9. A readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the handwriting recognition method according to any one of claims 1-4.

Images