An online handwritten Chinese character recognition method and system integrating stroke and structure information

Abstract

The present invention relates to an online handwritten Chinese character recognition method and system that incorporates stroke and structural information, and belongs to the field of pattern recognition and artificial intelligence. The present invention learns the temporal features in a coordinate point sequence through a long short-term memory network, finds the inflection point of the coordinate sequence through the relationship between adjacent points in the coordinate point sequence, regards the coordinate point between two consecutive inflection points as a coordinate point corresponding to a stroke, processes it to form a stroke feature, and sends the stroke feature to the network to learn the stroke sequence feature, divides the feature sequence into two parts in the time dimension, processes each part to form a structural feature, and sends it to a classifier for classification and recognition. The present invention takes into account the temporal information in the coordinate point sequence, and incorporates the core feature information inherent in Chinese characters such as strokes and structures, which can improve the accuracy of online handwritten Chinese character recognition to a certain extent; it helps the robustness of handwritten Chinese character recognition and the characteristics of small sample learning.

Description

Technical Field

[0001] The invention belongs to the field of pattern recognition and artificial intelligence, and specifically relates to an online handwritten Chinese character recognition method and system integrating stroke and structure information. Background Art

[0002] At present, the method for online handwritten Chinese character recognition mainly recognizes the coordinate point sequence and has achieved quite good results. However, this method based on the coordinate point sequence rarely takes the stroke and structure information contained in the coordinate point sequence into consideration, and thus has the following shortcomings:

[0003] (1) Based on the recognition algorithm at the coordinate point sequence level, the learned neural network model will mostly depend on the data type of the training data. For example, each person has different writing habits. The same stroke may be written by different people with different degrees of curvature and speed. This will result in differences in the position and number of coordinate points in the coordinate point sequence collected corresponding to the stroke. When the training data is insufficient, the model cannot learn the most basic features of the stroke from the training data and cannot recognize unknown test data.

[0004] (2) Stroke and structure information are the core features of Chinese characters. The recognition algorithm based on the coordinate point sequence level ignores the information of strokes and structure to a certain extent, resulting in the low degree of interpretability of the algorithm model. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is how to provide an online handwritten Chinese character recognition method and system that incorporates stroke and structure information to solve the problems that when the training data is insufficient, the model cannot learn the most basic features of the strokes in the training data, cannot recognize when faced with unknown test data, and the algorithm model has a low degree of interpretability.

[0007] (II) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes an online handwritten Chinese character recognition method integrating stroke and structure information, the method comprising the following steps:

[0009] S1. In the coordinate point sequence learning stage, the time series features in the coordinate point sequence are learned through the Long Short-Term Memory (LSTM) network;

[0010] S2. In the stroke sequence learning stage, the inflection point of the coordinate sequence is found through the relationship between adjacent points in the coordinate point sequence, and the coordinate point between two consecutive inflection points is regarded as the coordinate point corresponding to a stroke, which is processed to form stroke features, and the stroke features are sent to the long short-term memory network to learn the stroke sequence features;

[0011] S3, in the structural feature learning stage, the stroke sequence features output in the stroke sequence learning stage are evenly divided into two parts in the time dimension, and each part is processed to form a structural feature;

[0012] S4: Process the structural features and send them to the softmax classifier for classification and recognition.

[0013] Furthermore, the step S1 specifically includes: given an input time series x=(x 1 ,x 2 ,…,x t ,…,x T ), where x t is a column vector, which includes the horizontal and vertical coordinates m t ,n t , the vector x at each time moment of the input time series t Send it to the recursive neural network for calculation to obtain a series of hidden layer state vectors. At each moment, the neural network calculation process is as follows:

[0014]

[0015]

[0016] Among them, h 0 represents the initial state of the hidden layer state vector, h 0 =0,h 1 represents the hidden layer state vector at time 1, h t represents the hidden layer state vector at time t, represents the function of calculating the hidden layer state, θ represents the parameters of the corresponding neural network; after T iterations, T hidden layer states h = (h 1 ,h 2 ,…,h t ,…,h T ), which is the positive time series feature extracted by the unidirectional recurrent neural network; the recurrent neural network is a long short-term memory network.

[0017] Furthermore, before step S1, it also includes sampling and preprocessing steps, the sampling frequency is at the kHz level, and then the sampling points are preprocessed, and the horizontal, vertical distances between two points are ratioed to the length and width of the entire word. If the ratio is less than 0.02, one of the points is removed until the horizontal and vertical distances and length-width ratios of all two adjacent points are greater than 0.02, and the input time series is obtained.

[0018] Furthermore, the step S2 specifically includes: assuming that the current coordinate point x t (m t ,n t ), m t is the horizontal axis, n t is the ordinate, and the coordinates of the four adjacent coordinate points are, x t-1 (m t-1 ,n t-1 ), x t-2 (m t-2 ,n t-2 ), x t+1 (m t+1 ,n t+1 ), x t+2 (m t+2 ,n t+2 ), then calculate x according to the horizontal and vertical coordinates respectively t , x t-1 , x t+1 , three points at x t The angle θ 1 , and x t , x t-2 , x t+2 Three points at x t The angle θ 2 , if θ 1 and θ 2 If both are less than 90 degrees, then the current point is determined to be an inflection point and is determined as the boundary of each stroke.

[0019] Furthermore, the step S2 further includes: dividing the hidden layer state h=(h 1 ,h 2 ,…,h t ,…,h T ), T hidden layer states, assuming that by stroke segmentation, a total of I strokes are obtained, and the hidden layer states covered by each stroke are summed to obtain the input p = (p 1 ,p 2 ,…,p i ,…,p I );in, Indicates that the corresponding h in the i-th stroke is h=(h 1 ,h2 ,…,h t ,…,h T ), r represents the number of coordinate points contained in the i-th stroke; then p=(p 1 ,p 2 ,…,p i ,…,p I ) is sent to the Long Short-Term Memory network to learn and obtain the stroke sequence feature, q = (q 1 ,q 2 ,…,q i ,…,q I ).

[0020] Furthermore, the step S3 specifically includes:

[0021] The stroke sequence feature q = (q 1 ,q 2 ,…,q i ,…,q I ), is evenly divided into two parts in the time dimension, These two parts are sent to the Long Short-Term Memory network for learning, and we get Then the two parts are added together to obtain the online handwritten Chinese character structure features.

[0022] The present invention provides an online handwritten Chinese character recognition system incorporating stroke and structure information, the system comprising the following modules:

[0023] Coordinate point sequence temporal information module: learns the temporal features in the coordinate point sequence based on the Long Short-Term Memory (LSTM) network;

[0024] Stroke sequence extraction module: find the inflection point of the coordinate sequence through the relationship between adjacent coordinate points, determine the boundary of each stroke, and then extract the stroke sequence;

[0025] Stroke sequence feature learning module: learns the stroke sequence features in the stroke sequence based on the Long Short-Term Memory (LSTM) network;

[0026] Structural feature learning module: The stroke sequence features output by the stroke sequence feature learning module are processed in blocks in the time dimension to form structural features;

[0027] Decision-making module: Classification is performed based on the learned structural features of Chinese characters, and the classification loss adopts the Softmax loss function.

[0028] Furthermore, the coordinate point sequence time sequence information module learns the time sequence features in the stroke sequence based on the Long Short-Term Memory (LSTM) network;

[0029] Given an input time series x=(x 1 ,x 2 ,…,x t ,…,x T ), where x t is a column vector, which includes the horizontal and vertical coordinates m t ,n t , the vector x at each time moment of the input time series t Send it to the recursive neural network for calculation to obtain a series of hidden layer state vectors. At each moment, the neural network calculation process is as follows:

[0030]

[0031]

[0032] Among them, h 0 represents the initial state of the hidden layer state vector, h 0 =0,h 1 represents the hidden layer state vector at time 1, h t represents the hidden layer state vector at time t, represents the function of calculating the hidden layer state, θ represents the parameters of the corresponding neural network; after T iterations, T hidden layer states h = (h 1 ,h 2 ,…,h t ,…,h T ), which is the positive time series feature extracted by the unidirectional recurrent neural network.

[0033] Furthermore, the stroke sequence extraction module uses the relationship between adjacent coordinate points to find the inflection point of the coordinate sequence, determine the boundary of each stroke, and then extract the stroke sequence, which specifically includes:

[0034] Assume that the current coordinate point x t (m t ,n t ), m t is the horizontal axis, n t is the ordinate, and the coordinates of the four adjacent coordinate points are, x t-1 (m t-1 ,nt-1 ), x t-2 (m t-2 ,n t-2 ), x t+1 (m t+1 ,n t+1 ), x t+2 (m t+2 ,n t+2 ), then calculate x according to the horizontal and vertical coordinates respectively t , x t-1 , x t+1 , three points at x t The angle θ 1 , and x t , x t-2 , x t+2 Three points at x t The angle θ 2 , if θ 1 and θ 2 If the angles are all less than 90 degrees, the current point is determined to be an inflection point and is determined as the boundary of each stroke;

[0035] The stroke sequence feature learning module determines the stroke boundary based on the stroke sequence extraction module, and divides the coordinate point sequence output by the temporal feature learning module h = (h 1 ,h 2 ,…,h t ,…,h T ), T hidden layer states, assuming that a total of I strokes are obtained through stroke segmentation, and the hidden layer states covered by each stroke are summed, then the input p of the stroke sequence feature learning module is (p 1 ,p 2 ,…,p i ,…,p I );

[0036] in, Indicates that the corresponding h in the i-th stroke is h=(h 1 ,h 2 ,…,h t ,…,h T ), r represents the number of coordinate points contained in the i-th stroke;

[0037] Then p=(p 1 ,p 2 ,…,p i ,…,p I ) is sent to the Long Short-Term Memory network to learn and obtain the stroke sequence feature, q = (q 1 ,q 2 ,…,qi ,…,q I ).

[0038] Furthermore, the structural feature learning module converts the stroke sequence feature q=(q 1 ,q 2 ,…,q i ,…,q I ), is evenly divided into two parts in the time dimension, These two parts are sent to the Long Short-Term Memory network for learning, and we get

[0039] Then the two parts are added together to obtain the online handwritten Chinese character structure features output by the structure feature learning module.

[0040] (III) Beneficial effects

[0041] The present invention proposes an online handwritten Chinese character recognition method and system integrating stroke and structure information. The beneficial effects of the present invention are:

[0042] (1) The present invention proposes a handwritten Chinese character recognition method that incorporates stroke and structure information. This method introduces structural features such as strokes in the process of online handwritten Chinese character recognition based on a coordinate point sequence. It not only considers the temporal information in the coordinate point sequence, but also incorporates the stroke and structure information inherent in Chinese characters. This method can improve the accuracy of online handwritten Chinese character recognition to a certain extent.

[0043] (2) The handwritten Chinese character recognition method proposed by the present invention, which incorporates stroke and structure information, helps to improve the robustness of handwritten Chinese character recognition and the characteristics of small sample learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the overall architecture diagram of the online handwritten Chinese character recognition integrating stroke and structure information of the present invention;

[0045] Figure 2 Schematic diagram of inflection point extraction of coordinate sequence. DETAILED DESCRIPTION

[0046] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0047] In view of the problems existing in the existing handwriting recognition technology based on the coordinate point sequence level, the purpose of the present invention is to provide a handwritten Chinese character recognition method that integrates stroke and structural information. The handwriting recognition technology based on the coordinate point sequence level mainly directly sends the coordinate point sequence into a recursive neural network or a time-series convolutional neural network, directly learns the characteristics of handwritten Chinese characters through the sequence, and then classifies and recognizes the coordinate point sequence of the input handwritten Chinese character sample; the online handwritten Chinese character recognition method that integrates stroke and structural information also directly sends the coordinate point sequence into the recursive neural network. The difference is that after a certain degree of learning of the coordinate point sequence, the inflection point is judged by the relationship between adjacent points in the coordinate point sequence, and the output between adjacent inflection points of the recursive neural network is used as a stroke feature, and then all the stroke features are sent to the recursive neural network in time sequence, and then the feature sequence output by the stroke sequence is evenly divided into two parts in the time dimension, and each part is combined to form a structural feature, which is then sent to the classifier for classification.

[0048] The method proposed in the present invention not only retains the advantage of extracting natural time series features from coordinate point sequence in the coordinate point sequence recognition method, but also incorporates the core feature information inherent in Chinese characters such as strokes and structures, which can improve the robustness of handwritten Chinese character recognition and the characteristics of small sample learning to a certain extent. The present invention is mainly aimed at online handwritten Chinese character recognition tasks, and can also provide ideas for building network models for the recognition of other time series data.

[0049] The technical solution used by the present invention to solve the related technical problems is: an online handwritten Chinese character recognition method and system integrating stroke and structure information. The method includes four stages: coordinate point sequence learning, stroke sequence learning, structure feature learning and classification decision.

[0050] S1. In the coordinate point sequence learning stage, the temporal features in the coordinate point sequence are mainly learned through the Long Short-Term Memory (LSTM) network.

[0051] S2. In the stroke sequence learning stage, the inflection points of the coordinate sequence are found through the relationship between adjacent points in the coordinate point sequence, and the coordinate points between two consecutive inflection points are regarded as the coordinate points corresponding to a stroke. The stroke features are processed to form stroke features, and the stroke features are sent to the Long Short-Term Memory (LSTM) network to learn the stroke sequence features.

[0052] S3. In the structural feature learning stage, the stroke sequence features outputted in the stroke sequence learning stage are evenly divided into two parts in the time dimension, and each part is processed to form structural features.

[0053] S4: Process the structural features and send them to the softmax classifier for classification and recognition.

[0054] Figure 1 It is the overall architecture of the online handwritten Chinese character recognition system that integrates stroke and structure information, including the following modules:

[0055] (1) Coordinate point sequence temporal information module: mainly based on the long short-term memory network (LSTM) to learn the temporal features in the coordinate point sequence.

[0056] (2) The stroke sequence extraction module mainly finds the inflection point of the coordinate sequence through the relationship between adjacent coordinate points, determines the boundary of each stroke, and then extracts the stroke sequence.

[0057] (3) The stroke sequence feature learning module mainly learns the stroke sequence features in the stroke sequence based on the Long Short-Term Memory (LSTM) network.

[0058] (4) The structural feature learning module mainly processes the stroke sequence features output by the stroke sequence feature learning module in the time dimension to form structural features.

[0059] (5) Decision-making module: The decision-making module mainly performs classification based on the learned structural features of Chinese characters, and the classification loss is proposed to adopt the Softmax loss function.

[0060] Figure 1 This is the overall architecture diagram of online handwritten Chinese character recognition that integrates stroke and structural information. It mainly provides the detailed design of each module and the interaction diagram of the online handwritten Chinese character recognition method that integrates stroke and structural information based on the data flow.

[0061] Figure 1 The overall architecture diagram of online handwritten Chinese character recognition that integrates stroke and structure information mainly includes five core modules: coordinate point sequence time series information module, stroke sequence extraction module, stroke sequence feature learning module, structure feature learning module and decision judgment. The specific method includes the following steps:

[0062] 1. Coordinate point sequence timing information module:

[0063] It is mainly based on the Long Short-Term Memory (LSTM) network to learn the temporal features in the stroke sequence.

[0064] Given an input time series x=(x 1 ,x 2 ,…,x t ,…,x T), where x t is a column vector, which includes the horizontal and vertical coordinates m t ,n t , the vector x at each time moment of the input time series t Send it to the recursive neural network for calculation to obtain a series of hidden layer state vectors. At each moment, the neural network calculation process is as follows:

[0065]

[0066]

[0067] Among them, h 0 represents the initial state of the hidden layer state vector, h 0 =0,h 1 represents the hidden layer state vector at time 1, h t represents the hidden layer state vector at time t, represents the function of calculating the hidden layer state, and θ represents the parameters of the corresponding neural network. After T iterations, T hidden layer states h = (h 1 ,h 2 ,…,h t ,…,h T ), which is the positive time series feature extracted by the unidirectional recurrent neural network.

[0068] In the present invention, the unidirectional recurrent neural network is of the Long Short-Term Memory type and related variant types.

[0069] Before step S1, the method also includes sampling and preprocessing steps, where the sampling frequency is at the kHz level. The sampling points are then preprocessed, and the horizontal, vertical and horizontal distances between two points are ratioed to the length and width of the entire word. If the ratio is less than 0.02, one of the points is removed until the horizontal and vertical distances and the length-width ratios of all two adjacent points are greater than 0.02, thereby obtaining the input time series.

[0070] 2. Stroke sequence extraction module

[0071] Stroke sequence extraction module: This module mainly uses the relationship between adjacent coordinate points to find the inflection point of the coordinate sequence, determine the boundary of each stroke, and then extract the stroke sequence. The main process is as follows:

[0072] Assume that the current coordinate point x t (m t ,n t ), m t is the horizontal axis, n t is the ordinate, and the coordinates of the four adjacent coordinate points are, xt-1 (m t-1 ,n t-1 ), x t-2 (m t-2 ,n t-2 ), x t+1 (m t+1 ,n t+1 ), x t+2 (m t+2 ,n t+2 ), then calculate x according to the horizontal and vertical coordinates respectively t , x t-1 , x t+1 , three points at x t The angle θ 1 , and x t , x t-2 , x t+2 Three points at x t The angle θ 2 , if θ 1 and θ 2 If all are less than 90 degrees, then the current point is determined to be an inflection point and is determined as the boundary of each stroke, such as Figure 2 shown.

[0073] 3. Stroke sequence feature learning module

[0074] The stroke sequence feature learning module mainly determines the stroke boundary based on the stroke sequence extraction module, and divides the coordinate point sequence output by the temporal feature learning module h = (h 1 ,h 2 ,…,h t ,…,h T ), T hidden layer states, assuming that a total of I strokes are obtained through stroke segmentation, and the hidden layer states covered by each stroke are summed, then the input p of the stroke sequence feature learning module is (p 1 ,p 2 ,…,p i ,…,p I ).

[0075] in, Indicates that the corresponding h in the i-th stroke is h=(h 1 ,h 2 ,…,h t ,…,h T ), r represents the number of coordinate points contained in the i-th stroke.

[0076] Then p=(p 1 ,p 2 ,…,p i ,…,p I) is sent to the Long Short-Term Memory network to learn and obtain the stroke sequence feature, q = (q 1 ,q 2 ,…,q i ,…,q I ).

[0077] 4. Structural feature learning module

[0078] The structural feature learning module mainly transforms the stroke sequence feature q = (q 1 ,q 2 ,…,q i ,…,q I ), is evenly divided into two parts in the time dimension, These two parts are sent to the Long Short-Term Memory network for learning, and we get

[0079] Then the two parts are added together to obtain the online handwritten Chinese character structure features output by the structure feature learning module.

[0080] 4. Classification decision module

[0081] The classification decision module mainly uses the sofmax classifier to make classification decisions through the learned online handwritten Chinese character structural features that integrate stroke and structural information.

[0082] The beneficial effects of the present invention are:

[0083] (1) The present invention proposes a handwritten Chinese character recognition method that incorporates stroke and structure information. This method introduces structural features such as strokes in the process of online handwritten Chinese character recognition based on a coordinate point sequence. It not only considers the temporal information in the coordinate point sequence, but also incorporates the stroke and structure information inherent in Chinese characters. This method can improve the accuracy of online handwritten Chinese character recognition to a certain extent.

[0084] (2) The handwritten Chinese character recognition method proposed by the present invention, which incorporates stroke and structure information, helps to improve the robustness of handwritten Chinese character recognition and the characteristics of small sample learning.

[0085] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An online handwritten Chinese character recognition method that incorporates stroke and structure information. It is characterized in that The method comprises the following steps: S1. In the coordinate point sequence learning stage, the temporal features in the coordinate point sequence are learned through the long short-term memory network; S2. In the stroke sequence learning stage, the inflection point of the coordinate sequence is found through the relationship between adjacent points in the coordinate point sequence, and the coordinate point between two consecutive inflection points is regarded as the coordinate point corresponding to a stroke, which is processed to form stroke features, and the stroke features are sent to the long short-term memory network to learn the stroke sequence features; S3, in the structural feature learning stage, the stroke sequence features output in the stroke sequence learning stage are evenly divided into two parts in the time dimension, and each part is processed to form a structural feature; S4, process the structural features and send them to the softmax classifier for classification and recognition; in, The step S2 specifically includes: assuming that the current coordinate point x t (m t ,n t ), m t is the horizontal axis, n t is the ordinate, and the coordinates of the four adjacent coordinate points are, x t-1 (m t-1 ,n t-1 ), x t-2 (m t-2 ,n t-2 ), x t+1 (m t+1 ,n t+1 ), x t+2 (m t+2 ,n t+2 ), then calculate x according to the horizontal and vertical coordinates respectively t , x t-1 , x t+1 , three points at x t The angle θ 1 , and x t , x t-2 , x t+2 Three points at x t The angle θ 2 , if θ 1 and θ 2 If the angles are all less than 90 degrees, the current point is determined to be an inflection point and is determined as the boundary of each stroke; The step S2 further includes: dividing the hidden layer state h=(h 1 ,h 2 ,…,h t ,…,h T ), T hidden layer states, assuming that by stroke segmentation, a total of I strokes are obtained, and the hidden layer states covered by each stroke are summed to obtain the input p = (p 1 ,p 2 ,…,p i ,…,p I );in, Indicates that the corresponding h in the i-th stroke is h=(h 1 ,h 2 ,…,h t ,…,h T ), r represents the number of coordinate points contained in the i-th stroke; then p=(p 1 ,p 2 ,…,p i ,…,p I ) is sent to the long short-term memory network for learning, and the stroke sequence features are obtained, q = (q 1 ,q 2 ,…,q i ,…,q I ); The step S3 specifically includes: The stroke sequence feature q = (q 1 ,q 2 ,…,q i ,…,q I ), is evenly divided into two parts in the time dimension, Send these two parts to the long short-term memory network for learning, and get Then the two parts are added together to obtain the online handwritten Chinese character structure features.

2. The online handwritten Chinese character recognition method incorporating stroke and structure information as claimed in claim 1, It is characterized in that The step S1 specifically includes: given an input time series x=(x 1 ,x 2 ,…,x t ,…,x T ), where x t is a column vector, which includes the horizontal and vertical coordinates m t ,n t , the vector x at each time moment of the input time series t Send it to the recursive neural network for calculation to obtain a series of hidden layer state vectors. At each moment, the neural network calculation process is as follows: Among them, h 0 represents the initial state of the hidden layer state vector, h 0 =0,h 1 represents the hidden layer state vector at time 1, h t represents the hidden layer state vector at time t, represents the function of calculating the hidden layer state, θ represents the parameters of the corresponding neural network; after T iterations, T hidden layer states h = (h 1 ,h 2 ,…,h t ,…,h T ), which is the positive time series feature extracted by the unidirectional recurrent neural network; the recurrent neural network is a long short-term memory network.

3. The online handwritten Chinese character recognition method incorporating stroke and structure information as claimed in claim 2, It is characterized in that Before step S1, the method also includes sampling and preprocessing steps, where the sampling frequency is at the kHz level. The sampling points are then preprocessed, and the horizontal, vertical and horizontal distances between two points are ratioed to the length and width of the entire word. If the ratio is less than 0.02, one of the points is removed until the horizontal and vertical distances and the length-width ratios of all two adjacent points are greater than 0.02, thereby obtaining the input time series.

4. An online handwritten Chinese character recognition system that incorporates stroke and structure information. It is characterized in that The system includes the following modules: Coordinate point sequence temporal information module: learns the temporal features in the coordinate point sequence based on the long short-term memory network; Stroke sequence extraction module: find the inflection point of the coordinate sequence through the relationship between adjacent coordinate points, determine the boundary of each stroke, and then extract the stroke sequence; Stroke sequence feature learning module: learns the stroke sequence features in the stroke sequence based on the long short-term memory network; Structural feature learning module: The stroke sequence features output by the stroke sequence feature learning module are processed in blocks in the time dimension to form structural features; Decision-making module: Classification is performed based on the learned structural features of Chinese characters, and the classification loss adopts the Softmax loss function; in, The stroke sequence extraction module uses the relationship between adjacent coordinate points to find the inflection point of the coordinate sequence, determine the boundary of each stroke, and then extract the stroke sequence, which specifically includes: Assume that the current coordinate point x t (m t ,n t ), m t is the horizontal axis, n t is the ordinate, and the coordinates of the four adjacent coordinate points are, x t-1 (m t-1 ,n t-1 ), x t-2 (m t-2 ,n t-2 ), x t+1 (m t+1 ,n t+1 ), x t+2 (m t+2 ,n t+2 ), then calculate x according to the horizontal and vertical coordinates respectively t , x t-1 , x t+1 , three points at x t The angle θ 1 , and x t , x t-2 , x t+2 Three points at x t The angle θ 2 , if θ 1 and θ 2 If the angles are all less than 90 degrees, the current point is determined to be an inflection point and is determined as the boundary of each stroke; The stroke sequence feature learning module determines the stroke boundary based on the stroke sequence extraction module, and divides the coordinate point sequence output by the temporal feature learning module into h=(h 1 ,h 2 ,…,h t ,…,h T ), T hidden layer states, assuming that a total of I strokes are obtained through stroke segmentation, and the hidden layer states covered by each stroke are summed, then the input p of the stroke sequence feature learning module is (p 1 ,p 2 ,…,p i ,…,p I ); in, Indicates that the corresponding h in the i-th stroke is h=(h 1 ,h 2 ,…,h t ,…,h T ), r represents the number of coordinate points contained in the i-th stroke; Then p=(p 1 ,p 2 ,…,p i ,…,p I ) is sent to the long short-term memory network for learning, and the stroke sequence features are obtained, q = (q 1 ,q 2 ,…,q i ,…,q I ); The structural feature learning module converts the stroke sequence feature q = (q 1 ,q 2 ,…,q i ,…,q I ), is evenly divided into two parts in the time dimension, Send these two parts to the long short-term memory network for learning, and get Then the two parts are added together to obtain the online handwritten Chinese character structure features output by the structure feature learning module.

5. The online handwritten Chinese character recognition system incorporating stroke and structure information as claimed in claim 4, It is characterized in that The coordinate point sequence time sequence information module learns the time sequence features in the stroke sequence based on the long short-term memory network; Given an input time series x=(x 1 ,x 2 ,…,x t ,…,x T ), where x t is a column vector, which includes the horizontal and vertical coordinates m t ,n t , the vector x at each time moment of the input time series t Send it to the recursive neural network for calculation to obtain a series of hidden layer state vectors. At each moment, the neural network calculation process is as follows: Among them, h 0 represents the initial state of the hidden layer state vector, h 0 =0,h 1 represents the hidden layer state vector at time 1, h t represents the hidden layer state vector at time t, represents the function of calculating the hidden layer state, θ represents the parameters of the corresponding neural network; after T iterations, T hidden layer states h = (h 1 ,h 2 ,…,h t ,…,h T ), which is the positive time series feature extracted by the unidirectional recurrent neural network.

Images