HANDWRITTEN TEXT RECOGNITION DEVICE SUPPORTING MULTIPLE WRITING DIRECTIONS.

Abstract

Handwriting text recognition device (1) supporting multiple writing directions, comprising a processor (2) and a memory (6); a stroke input processor (10) for receiving a plurality of strokes from inline handwriting character strings that are split into a plurality of strokes; a writing direction determination operator connected to the stroke input processor for computing a writing direction of strokes from inline handwriting character strings using a coordinate of a center of gravity and a plurality of second central moments of a plurality of stroke sampling points; a stroke sorting operator connected to the writing direction determination operator and sorting the strokes according to the writing direction to obtain a set of sort tracks labeled in the x direction or a set of sort tracks labeled in the y direction.Figure to be published with the abbreviation: Fig. 1.

Description

Title of the invention: HANDWRITTEN TEXT RECOGNITION APPARATUS SUPPORTING MULTIPLE WRITING DIRECTIONS.

[0001] FIELD OF THE INVENTION.

[0002] The invention relates to a text recognition system, and in particular to a handwritten text recognition apparatus supporting multiple writing directions.

[0003] CONTEXT OF THE INVENTION.

[0004] Most online handwriting recognition systems are designed to recognize texts with horizontal writing. However, languages ​​such as Chinese, Japanese, and Korean traditionally have a vertical writing style that is still used on certain occasions today. Sometimes, the horizontal writing direction and the vertical writing direction are even used in combination. In the case of Chinese Spring Festival couplets, the vertical writing direction is used for the right and left scrolls of the couplets, while the horizontal writing direction is used for the upper scroll of the couplets. Therefore, it would be useful to have an online handwriting text recognition system that supports both the horizontal and vertical writing directions without requiring the user to manually switch between the recognition modes.Although a prior art superimposed handwritten text recognition model supports multidirectional writing, it does not support handwritten texts written with strokes whose order is reversed at the same time.

[0005] Furthermore, in the optical text recognition technology of the prior arts, a classifier is generally used to distinguish horizontal and vertical writing directions in input texts, and then the horizontal and vertical writing directions are processed separately, for example, by rotating the vertically written text. However, the above method will encounter difficulties when used to further process other writing directions.

[0006] SUMMARY OF THE INVENTION.

[0007] Accordingly, to improve the above-mentioned defects of the prior art, the object of the invention is to provide a handwritten text recognition apparatus supporting multiple writing directions, wherein the advantages of the invention are that the invention can recognize strokes of handwritten texts written in different directions as a character string, and can support any written handwritten text whose strokes are written in an arbitrary or reversed order. It should be noted that there is no need to specifically collect vertical handwritten texts during the training phase of the invention, which eliminates the cost of collecting and labeling data samples. The invention achieves higher recognition accuracy for handwritten texts with different writing directions, and provides the corresponding relationships between input strokes and output characters. The invention also enables the recognition of any handwritten text whose strokes are written in an arbitrary or reversed order.

[0008] BRIEF DESCRIPTION OF THE FIGURES.

[0009] [Fig. 1] is a structured functional diagram showing the elements of the training phase of the recognition model of the invention.

[0010] [Fig.2] is a structured functional diagram showing the elements of the prediction phase of the recognition model of the invention.

[0011] [Fig.3] is a flow diagram of steps showing the recognition of the invention.

[0012] [Fig.4] is a flow diagram of steps showing the generation of samples learning about invention.

[0013] [Fig.5] is a structured block diagram showing the elements of the handwritten text recognition apparatus of the invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] Figures 1 to 5 illustrate a handwritten text recognition apparatus 1 supporting multiple writing directions according to the invention. The handwritten text recognition apparatus 1 comprises a processor 2 and a memory 6, as shown in [Fig. 5]. The processor 2 is used to execute the required operations of the handwritten text recognition apparatus 1 and the memory 6 stores the data, programs and results of the related operations of the handwritten text recognition apparatus 1.

[0015] In the invention, the recognition of handwritten texts is modeled as a sequence-to-sequence model. The input of the model is a sequence of sample points (including x-coordinates, y-coordinates, and stroke end tokens) expressing the strokes of the handwritten texts, while the output of the model is a character string (a character sequence) corresponding to the strokes. The lengths (numbers) of the input strokes and the output character string are not fixed. The invention provides a recognition model 500 which is a neural network based on an encoder-decoder model for the recognition of handwritten texts. The handwritten text recognition apparatus 1 further comprises the following elements.

[0016] A stroke input processor 10 is used to receive a plurality of online handwriting character string tracks 80 from an electronic handwriting input device 600 (referring to step 100 of [Fig. 3]). The online handwriting character string tracks 80 are split into a plurality of strokes 81. Each of the strokes 81 includes a plurality of sampling points 82. The stroke input processor 10 preprocesses the strokes 81 of the online handwriting character string tracks 80 in a training phase and a prediction phase of the neural network of the invention.

[0017] A writing direction determining operator 20 is connected to the stroke input processor 10 to calculate a writing direction of the online handwriting character string tracks 80 for identifying a direction of an online handwriting character string (referring to step 110 of [Fig. 3]). In the invention, the sorting of the strokes 81 is used as a preprocessing step to ensure that the recognition results are independent of the order of the strokes 81. The sorting of the strokes 81 is based on the writing direction calculated by the writing direction determining operator 20. The writing direction of the online handwriting character string tracks 80 is calculated according to the following method.

[0018] The sampling points 82 of each of the strokes 81 are resampled before calculating the writing direction so that the distances between the two adjacent points of the same stroke 81 are approximately the same.

[0019] When calculating the writing direction, the number of strokes 81 is defined as m. The nth stroke i 81 of the strokes 81 is composed of ni sampling points 82 which are represented as (x_(i, 1), y_(i,l)), (x_(i,2), y_(i,2)), (x_ (i, ni), y_ (i, ni)). The total number of sampling points 82 of the strokes 81 is n=E_(i=l) Amini. A coordinate of the center of gravity of the sampling points 82 of the strokes 81 is (x ", y “), in which x “=l / n E_(i=l) Am^E_(j=l) A(n_i) -Kx (i, j), y “=l / n E_(i= 1) Am^E_(j=l) A(n_i) -Ky (i, j), j= 1, n_i. The three second central moments of the sampling points 82 of the strokes 81 are IX=l / n E_(i=l) Am^E_(j=l) A(n_i) (x_ (i, j) -x ") A2,I_xy=l / n E_(i=l) M£E_(j=l) A(n_i) (x_ (i, j) -x ") (y_ (i, j) -y “), I_yy=l / n E_(i=l) Am^E_(j=l) A(n_i) (y_ (i, j) -y “) A2.An angle 0 between an x-axis and a line that minimizes the sum of the squares of the distances between each of the 82 sample points and the line is 0_l=l / 2 "arctan" ((2I_xy) / (I_xx-I_yy)) or 0_2=0_l+ir / 2. The writing direction is the angle 0=0_k, in which k= (argmax() -r(k= 1,2) I_xx cosA2(0_k) +2I_xy cos(0_k) sin(0_k) +I_yy sinA2(0_k)). .

[0020] The calculated writing direction (represented by angle 0) is used as the estimated writing direction of the online handwriting character string.

[0021] A stroke sorting operator 30 is connected to the writing direction determination operator 20. The stroke sorting operator 30 sorts the strokes 81 according to the writing direction calculated by the writing direction determination operator 20, as illustrated in [Fig.2]. The writing direction is represented by the angle 0.When the angle 0 represented by the writing direction is between -ji / 4 and ji / 4, that is, a direction indicated by the writing direction is closer to a horizontal axis (x-axis) than to a vertical axis (y-axis) and the inline handwriting character string tracks 80 tend to have a horizontal writing direction, the stroke sorting operator 30 sorts the strokes 81 in an order from the stroke 81 having the smallest x-coordinate to the stroke 81 having the largest x-coordinate (from top to bottom) so as to obtain a set of sorting strokes labeled in the x-direction 31 (referring to step 120 of [Fig. 3]).

[0022] When the angle 0 represented by the writing direction is between ir / 4 and 3ir / 4, that is, the direction indicated by the writing direction is closer to the vertical axis (y-axis) than to the horizontal axis (x-axis), the inline handwriting character string 80 tends to have a vertical writing direction, the stroke sorting operator 30 sorts the strokes 81 in an order from the stroke 81 having the smallest y-coordinate to the stroke 81 having the largest y-coordinate (from left to right) so as to obtain a set of sorting strokes labeled in the y-direction 32 (referring to step 130 of [Fig. 3]).

[0023] For handwriting character strings with different writing directions, strokes 81 belonging to a single character tend to be grouped by the sorting of the stroke sorting operator 30. Therefore, each character in the inline handwriting character string tracks 80 can be correctly split and recognized in subsequent recognition.

[0024] A training sample generator 40 is connected to the writing direction determination operator 20 and the stroke sorting operator 30. The training sample generator 40 is used to generate a plurality of training samples for training the neural network during the training phase, as illustrated in [Fig.l]. The invention uses synthetic training samples to reduce the work of collecting additional handwriting samples.

[0025] An angle selector 43 is connected to the training sample generator 40. The angle selector 43 serves to provide a training angle as the writing direction of the training sample.

[0026] The training sample generator 40 comprises a corpus 41, a database of unique handwritten online characters 42 and a database of training character strings 45. The corpus 41 is composed of texts in natural language. The corpus 41 is composed of a plurality of natural language texts, which is a well-compiled balanced corpus or a corpus of web texts collected by a web crawler. The online handwritten unique character database 42 comprises a plurality of handwriting track sets and each of the handwriting track sets comprises a plurality of handwriting strokes corresponding to a unique character.

[0027] With reference to [Fig.4], when generating the training sample, the training sample is generated by following the following steps:

[0028] Step 200: The training sample generator 40 randomly selects a text of a length within a determined range from the corpus 41 as a training text. The training text is composed of a character string comprising a plurality of characters.

[0029] Step 210: The training sample generator 40 selects a training angle 0_0 between -ji / 4 and ir / 4 or between ir / 4 and 3ir / 4 from the angle selector 43, and establishes a baseline L with the training angle 0_0 as the inclination angle of the baseline L. When the training angle 0_0 selected by the training sample generator 40 is between -ji / 4 and ji / 4, the baseline L is considered a horizontal line. When the training angle 0_0 selected by the training sample generator 40 is between ir / 4 and 3ir / 4, the baseline L is considered a vertical line.

[0030] Step 220: The training sample generator 40 finds the handwriting track set corresponding to the 1st character of the training text in the online handwritten single character database 42, and moves the handwriting track set corresponding to the 1st character to a starting point of the baseline L. A center of the handwriting track set corresponding to the 1st character is located at the starting point of the baseline L. The handwriting track set corresponding to the 1st character is distributed along a direction of a stroke of the baseline L.

[0031] Step 230: For each remaining character of the training text, the training sample generator 40 finds the corresponding handwriting track set in the online handwritten single character database 42, and moves each of the corresponding handwriting track sets to a position located after a previous handwriting track set on the baseline L. The center of each of the handwriting track sets positioned on the baseline L is located on the baseline L. The distances between the centers of two adjacent handwriting track sets on the baseline L obey a normal distribution.

[0032] Step 240: The training sample generator 40 slightly distorts the writing strokes of all the sets of writing tracks positioned on the baseline L in order to obtain a set of simulated character string tracks. The set of tracks of the simulated character string is a plurality of strokes of a plurality of strokes corresponding to the character string of the training text.

[0033] Step 250: The writing direction of the simulated character string is calculated by the writing direction determination operator 20 (because the writing direction of the simulated character string will be slightly changed to be different from the learning angle 0_0 during the deformation in the previous step 240). When the writing direction of the simulated character string track set is between -ji / 4 and ji / 4, the stroke sorting operator 30 sorts the strokes of the simulated character string track set according to the x coordinates of the sampling points of the strokes of the simulated character string track set to obtain a horizontal synthetic track set 46.When the writing direction of the simulated string track set is between ir / 4 and 3ir / 4, the stroke sorting operator 30 sorts the strokes of the simulated string track set according to the y coordinates of the sample points of the strokes of the simulated string track set, so as to obtain a vertical synthetic track set 44. [Fig. 4] shows that the writing direction is between -ji / 4 and ji / 4, so that the stroke sorting operator 30 sorts the strokes of the simulated string track set according to the x coordinates of the sample points of the strokes of the simulated string track set to obtain a horizontal synthetic track set 46. .

[0034] Step 260: The set of horizontal synthetic tracks 46 or the set of vertical synthetic tracks 44 obtained by the stroke sorting operator 30 is used as a training sample and is stored in the training string database 45 by the training sample generator 40.

[0035] The training samples generated by the training sample generator 40 cover different writing directions, and these training samples have significant text content and some similarities with human handwriting. The training samples are used for training the recognition model 500. At the same time, in these training samples, a correspondence between each of the characters in the training text and the strokes in each of the handwriting track sets of the simulated character string track set is deterministic, which is useful for training the recognition model 500 and some other models, for example, it may be used for artificial neural networks to generate more accurate alignments.

[0036] In the invention, the recognition model 500 may be any sequence-to-sequence model trained by end-to-end learning. In the invention, the recognition model 500 is the text recognition neural network model 50 which is based on an encoder-decoder model.

[0037] The recognition model 500 is connected to the stroke sorting operator 30 or the training string database 45. The recognition model 500 is a text recognition neural network model 50. The text recognition neural network model 50 comprises an encoder 51 having an input interface 53, and a decoder 52 having an output interface 54. The input interface 53 is connected to the stroke sorting operator 30 or the training string database 45. The input interface 53 is used to receive stroke data from the stroke sorting operator 30 or the training string database 45.The stroke data includes the x-direction labeled sorting stroke set 31 or the y-direction labeled sorting stroke set 32 ​​of the stroke sorting operator 30, or the strokes of the horizontal synthetic track set 46 or the vertical synthetic track set 44 and a character string corresponding to the strokes of the training character string database 45. The text recognition neural network model 50 repeatedly recognizes and calculates the stroke data to obtain a plurality of possible character strings and reliability (referring to step 140 of [Fig. 3]). The possible character strings and their reliability are stored in a candidate set 70 which is output from the output interface 54 (referring to step 150 of [Fig. 3]).

[0038] Referring to [Fig. 1], in the training phase of the text recognition neural network model 50, the strokes of the horizontal synthetic track set 46 or the vertical synthetic track set 44 are used as input values ​​of the text recognition neural network model 50. The characters of the character string corresponding to the strokes of the horizontal synthetic track set 46 or the vertical synthetic track set 44 are used as predicted output values ​​of the text recognition neural network model 50.Therefore, supervised learning may be used to train the text recognition neural network model 50 using the horizontal synthetic track set 46 or the vertical synthetic track set 44 and the corresponding character string of the horizontal synthetic track set 46 or the vertical synthetic track set 44. The horizontal synthetic track set 46 or the synthetic track set . verticals 44 comprises handwriting tracks comprising sorted strokes in order to better simulate the input values ​​in the prediction phase of the text recognition neural network model 50.

[0039] Referring to [Fig.2], in the prediction phase of the text recognition neural network model 50, the input to the recognition model 500 is a sequence of sampling points 82 of the strokes 81. The writing direction of the strokes is calculated by the writing direction determination operator 20 and is transmitted to the stroke sorting operator 30 for acquiring the x-direction labeled sorting stroke set 31 or the y-direction labeled sorting stroke set 32, depending on whether the writing direction is closer to a horizontal axis than to a vertical axis. In the prediction phase of the text recognition neural network model 50, the sampling points of the strokes 81 of the x-direction labeled sorting stroke set 31 or the y-direction labeled sorting stroke set 32 ​​are used as input values ​​of the text recognition neural network model 50.The output values ​​of the text recognition neural network model 50 are a sequence of characters (a string of characters). .

[0040] Preferably, the training samples generated by the training sample generator 40 are based on text strokes written in a horizontal writing direction, a vertical writing direction, and other writing directions to ensure that the recognition model 500 can be trained by strokes sorted from these text strokes written in different writing directions.

[0041] During a testing phase of the recognition model 500, other sets of horizontal or vertical synthetic tracks can be generated in order to obtain recognition accuracy of the recognition model 500.

[0042] FOLLOWING OTHER FEATURES

[0043] The apparatus of the invention, wherein the recognition model 1 is a sequence-to-sequence model trained by end-to-end learning.

[0044] The apparatus of the invention, wherein the recognition model 1 is a text recognition neural network model 500 based on an encoder-decoder model; in a prediction phase of the text recognition neural network model, the sampling points of the strokes of the x-direction labeled sorting stroke set or the y-direction labeled sorting stroke set are used as input values ​​of the text recognition neural network model; the output values ​​of the text recognition neural network model are a character sequence.

[0045] The apparatus of the invention, further comprising:

[0046] a training sample generator connected to the writing direction determination operator and the stroke sorting operator; the training sample generator is for generating a plurality of training samples; an angle selector connected to the training sample generator; the angle selector is for providing a training angle; wherein the training sample generator comprises a corpus, an online single-character handwriting database, and a training character string database; the corpus is composed of natural language texts; the online single-character handwriting database comprises a plurality of handwriting track sets, and each of the handwriting track sets comprises a plurality of handwriting strokes corresponding to a single character;wherein, when generating the training sample, the training sample generator randomly selects a text of a length within a predetermined range from the corpus as the training text; the training text is composed of a character string comprising a plurality of characters; and the training sample generator selects a training angle between -ji / 4 and ir / 4 or between ir / 4 and 3ir / 4 from the angle selector, and establishes a baseline with the training angle as the inclination angle of the baseline; when the training angle selected by the training sample generator is between -ji / 4 and ji / 4, the baseline is considered a horizontal line; when the training angle selected by the training sample generator is between ir / 4 and 3ir / 4, the baseline is considered a vertical line;the training sample generator finds the handwriting track set corresponding to the 1st character of the training text in the online handwritten single character database, and moves the handwriting track set corresponding to the 1st character to a starting point of the baseline; a center of the handwriting track set corresponding to the 1st character is located at the starting point of the baseline; the handwriting track set corresponding to the 1st character is distributed along a direction of a track of the baseline;for each of the other characters of the training text, the training sample generator sequentially finds the corresponding handwriting track sets in the online handwritten single character database, and moves each of the corresponding handwriting track sets to a position located after a previous handwriting track set on the baseline track; a center of each of the handwriting track sets positioned on the baseline is located on the baseline track; the distances between two centers of both; adjacent sets of writing tracks on the baseline obey a normal distribution; and the training sample generator slightly distorts the writing tracks of all sets of writing tracks positioned on the baseline to obtain a simulated character string track set; the simulated character string track set is a plurality of tracks of a plurality of strokes corresponding to the character string of the training text; and the writing direction of the simulated character string track set is calculated by the writing direction determination operator;when the writing direction of the simulated string track set is between -ji / 4 and ji / 4, the stroke sorting operator sorts the strokes of the simulated string track set according to the x coordinates of the sample points of the strokes of the simulated string track set, so as to obtain a horizontal synthetic track set; while when the writing direction of the simulated string track set is between ir / 4 and 3ir / 4, the stroke sorting operator sorts the strokes of the simulated string track set according to the y coordinates of the sample points of the strokes of the simulated string track set, so as to obtain a vertical synthetic track set;the horizontal synthetic track set or the vertical synthetic track set obtained by the stroke sorting operator is used as a training sample and is stored in the training character string database by the training sample generator; and wherein the training samples generated by the training sample generator cover various writing directions and the correspondence between each of the characters in the training text and the strokes in each of the handwriting track sets of the simulated character string track set is deterministic; the training samples are used for training the recognition model. ;

[0047] The apparatus of the invention, wherein the recognition model is a text recognition neural network model; in the training phase of the text recognition neural network model, the strokes of the horizontal synthetic track set or the vertical synthetic track set are used as input values ​​of the text recognition neural network model; and the characters of the character string corresponding to the strokes of the horizontal synthetic track set or the vertical synthetic track set are used as expected output values ​​of the text recognition neural network model.

[0048] The apparatus of the invention, wherein the recognition model is a text recognition neural network model; the text recognition neural network model text recognition comprises an encoder which comprises an input interface, and a decoder which comprises an output interface; the input interface is connected to the stroke sorting operator or the training character string database; the input interface is used for receiving stroke data from the stroke sorting operator or the training character string database; the stroke data comprises the x-direction labeled sorting stroke set or the y-direction labeled sorting stroke set from the stroke sorting operator, or the strokes of the horizontal synthetic track set or the vertical synthetic track set and a corresponding character string from the training character string database;the text recognition neural network model repeatedly recognizes and calculates the stroke data to acquire a plurality of possible character strings and reliability; the possible character strings and reliability are stored in a candidate set which is output by the output interface. ;

[0049] The apparatus of the invention, wherein the sampling points of each of the strokes are resampled before calculating the writing direction so that the distances between the two adjacent points of the same stroke are approximately the same; and wherein, when calculating the writing direction, the number of strokes is defined as m; the ith stroke 81 of the strokes 81 is composed of n_i sampling points which are represented as (x_(i,l),y_(i,l) ),(x_(i,2),y_(i,2) ), ...,(x_(i,n_i ),y_(i,n_i ) ); the total number of feature sampling points is n=E_(i=DAm^n__i ; a coordinate of the center of gravity of the feature sampling points is (x ",y " ), in which x “=l / n E_(i=l)Am^E_(j=l)A(n_i)^x_(i,j) ,y “=1 / n E_(i=l)Ani^E_(j=l)A(n_i)^y_(i,j) ,j=l,...,n_i; the three second central moments of the sampling points 82 of the strokes 81 are I_xx=l / n E (i=l)Am^E (J=DAf n_i)^(x_(i,j)-x " )A2 ,I_xy=l / n E_(i=l)Am^E_(j=l)A(n_i)^(x_(i,j)-x " ) (y_(i,j)-y " ),I_yy=l / n E_(i=l)Am^E_(j=1)A(n_i)^(y_(i,j)-y " )A2 ; an angle 0 between an x-axis and a line that minimizes the sum of the squares of the distances between each of the 82 sample points and the line is 0_l=l / 2 "arctan" ((2I_xy) / (I_xx-I_yy )) or 0_2=0_l+ir / 2; and the writing direction is the angle 0=0_k, in which k=(argmax()-r(k=l,2) I_xx cosA2 (0_k )+2I_xy cos(0_k )sin(0_k )+I_yy sinA2 (0_k )). .

[0050] The apparatus of the invention, wherein the corpus is a balanced corpus or a corpus of web texts collected by a web indexing robot.

[0051] The elements of the learning phase of the recognition model of the invention are represented in the table below: 20 Operator for determining the writing direction 30 Operator for sorting strokes 40 Training Sample Generator 41 Corpus 42 Online Handwritten Unique Character Database 43 Angle Selector 44 Vertical Synthetic Track Set 45 Training String Database 46 Horizontal Synthetic Track Set 500 Text Recognition Neural Network Model 50 51 Encoder 52 Decoder 53 Input Interface 54 Output Interface 70 Candidate Set

[0052] The elements of the prediction phase of the recognition model are represented in the table below: 600 Electronic handwriting input device 80 Online handwriting character string tracks 81 Strokes 82 Sample points 83 Sample points 10 Stroke input processor 20 Writing direction determination operator 30 Stroke sorting operator r 31 Set of labeled sorting strokes in the x direction 32 Set of labeled sorting strokes in the y direction The flow of steps showing the recognition according to the invention is represented in the table below: Step 100 The stroke input processor 10 receives the handwriting character string tracks in line 80 Step 110 The writing direction determination operator 20 calculates the writing direction of the handwriting string tracks in line 80. Do the strokes 81 have a horizontal writing direction or a vertical writing direction based on the writing direction? Horizontal writing direction Step 120 The stroke sorting operator 30 sorts the strokes 81 from left to right. Vertical writing direction Step 130 The stroke sorting operator 30 sorts the strokes 81 from top to bottom. Step 140 Input the sorted strokes 81 to the recognition model 500 for recognition. Step 150 Output the candidate set 70.

[0053] The elements of the learning phase of the recognition model of the invention are represented in the table below:

[0054] The invention is thus described, but it is obvious that it can be modified in many ways. Such variations should not be construed as a departure from the spirit and scope of the invention, and all modifications which would be obvious to a person skilled in the art are intended to be included within the scope of the following claims.

Claims

1. Claims A handwritten text recognition device (1) supporting multiple writing directions, comprising: - a processor (2) and a memory (6); the processor being used for performing the required operations of the handwritten text recognition apparatus (1) and the memory storing the data, programs and results of related operations of the handwritten text recognition apparatus; - a stroke input processor (10) for receiving a plurality of tracks of online handwriting character strings from an electronic handwriting input device; the tracks of online handwriting character strings being split into a plurality of strokes (81); each of the strokes comprising a plurality of sample points; - a writing direction determining operator (30) connected to the stroke input processor for calculating a writing direction of the online handwriting character string tracks for recognizing (1) a direction of an online handwriting character string; the writing direction is represented by an angle 0 which is calculated using a coordinate of a center of gravity and a plurality of second center moments of the stroke sampling points; - a stroke sorting operator connected to the writing direction determination operator; the stroke sorting operator sorts the strokes according to the writing direction calculated by the writing direction determination operator; wherein, when the angle 0 represented by the writing direction is between -ji / 4 and ji / 4, i.e., a direction indicated by the writing direction is closer to a horizontal axis than to a vertical axis and the online handwriting character string tends to have a horizontal writing direction, the stroke sorting operator sorts the strokes in an order from the stroke with the smallest x-coordinate to the stroke with the largest x-coordinate, so

2.

3.

4. to obtain a set of sorting strokes labeled in the x direction; while when the angle 0 represented by the writing direction is between ir / 4 and 3ir / 4, i.e., the direction indicated by the writing direction is closer to the vertical axis than the horizontal axis and online handwriting character strings tend to have a vertical writing direction, the stroke sorting operator sorts the strokes in an order from the stroke with the smallest y coordinate to the stroke with the largest y coordinate, so as to obtain a set of sorting strokes labeled in the y direction; - in which the features belonging to a single character tend to be grouped together by sorting the feature sorting operator; and - wherein the set of labeled sorting strokes in the x direction or the set of labeled sorting strokes in the y direction is transmitted to a recognition model (500) for recognition. The apparatus of claim 1, wherein the recognition model is a sequence-to-sequence model trained by end-to-end learning. The apparatus of claim 1, wherein the recognition model (1) is a text recognition neural network model (500) based on an encoder-decoder model; in a prediction phase of the text recognition neural network model, the sample points of the strokes of the x-direction labeled sorting stroke set or the y-direction labeled sorting stroke set are used as input values ​​of the text recognition neural network model; the output values ​​of the text recognition neural network model are a character sequence. The apparatus of claim 1, further comprising: - a training sample generator connected to the writing direction determination operator and the stroke sorting operator; the training sample generator is for generating a plurality of training samples; - an angle selector connected to the training sample generator; the angle selector is used to provide a training angle; - wherein the training sample generator comprises a corpus, a database of character-based handwriting unique online and a training character string database; the corpus is composed of natural language texts; the online handwritten unique character database comprises a plurality of handwriting track sets and each of the handwriting track sets comprises a plurality of handwriting strokes corresponding to a unique character; - wherein, when generating the training sample, the training sample generator randomly selects a text of a length within a determined range from the corpus as the training text; the training text is composed of a character string comprising a plurality of characters; and the training sample generator selects a training angle between -ji / 4 and ir / 4 or between ir / 4 and 3ir / 4 from the angle selector, and establishes a baseline with the training angle as the inclination angle of the baseline; when the training angle selected by the training sample generator is between -ji / 4 and ji / 4, the baseline is considered a horizontal line; when the training angle selected by the training sample generator is between ir / 4 and 3ir / 4, the baseline is considered a vertical line;the training sample generator finds the handwriting track set corresponding to the 1st character of the training text in the online handwritten single character database, and moves the handwriting track set corresponding to the 1st character to a starting point of the baseline; a center of the handwriting track set corresponding to the 1st character is located at the starting point of the baseline; the handwriting track set corresponding to the 1st character is distributed along a direction of a track of the baseline;for each of the other characters in the training text, the training sample generator sequentially finds the corresponding handwriting track sets in the online handwritten single character database, and moves each of the corresponding handwriting track sets to a position after a previous handwriting track set on the baseline track; a center of each; handwriting track sets positioned on the baseline is located on the baseline track; the distances between two centers of every two adjacent writing track sets on the baseline obey a normal distribution; and the training sample generator slightly distorts the writing tracks of all the writing track sets positioned on the baseline to obtain a simulated character string track set; the simulated character string track set is a plurality of tracks of a plurality of strokes corresponding to the character string of the training text; and the writing direction of the simulated character string track set is calculated by the writing direction determination operator;when the writing direction of the simulated string track set is between -ji / 4 and ji / 4, the stroke sorting operator sorts the strokes of the simulated string track set according to the x coordinates of the sample points of the strokes of the simulated string track set, so as to obtain a horizontal synthetic track set; while when the writing direction of the simulated string track set is between ir / 4 and 3ir / 4, the stroke sorting operator sorts the strokes of the simulated string track set according to the y coordinates of the sample points of the strokes of the simulated string track set, so as to obtain a vertical synthetic track set;the set of horizontal synthetic tracks or the set of vertical synthetic tracks obtained by the stroke sorting operator is used as a training sample and is stored in the training string database by the training sample generator; and; - wherein the training samples generated by the training sample generator cover various writing directions and the correspondence between each of the characters in the training text and the strokes in each of the handwriting track sets of the simulated character string track set is deterministic; the training samples are used for training the recognition model.

5. The apparatus of claim 4, wherein the recognition model is a text recognition neural network model; in the training phase of the text recognition neural network model, the strokes of the horizontal synthetic track set or the vertical synthetic track set are used as input values ​​of the text recognition neural network model; and the characters of the character string corresponding to the strokes of the horizontal synthetic track set or the vertical synthetic track set are used as expected output values ​​of the text recognition neural network model.

6. The apparatus of claim 4, wherein the recognition model is a text recognition neural network model; the text recognition neural network model comprises an encoder which comprises an input interface, and a decoder which comprises an output interface; the input interface is connected to the stroke sorting operator or the training character string database; the input interface is for receiving stroke data from the stroke sorting operator or the training character string database;the stroke data comprises the x-direction labeled sorting stroke set or the y-direction labeled sorting stroke set from the stroke sorting operator, or the strokes of the horizontal synthetic track set or the vertical synthetic track set and a corresponding character string from the training character string database; the text recognition neural network model repeatedly recognizes and calculates the stroke data to acquire a plurality of possible character strings and reliability; the possible character strings and reliability are stored in a candidate set which is output by the output interface.;

7. The apparatus of claim 1, wherein the sampling points of each of the strokes are resampled before calculating the writing direction so that the distances between the two adjacent points of the same stroke are approximately the same; and

8. - wherein, when calculating the writing direction, the number of strokes is defined as m; the nth stroke 81 of the strokes 81 is composed of ni sampling points which are represented as (x_(i,l),y_(i,l) ),(x_(i,2),y_(i,2) ),..., (x_(i,n_i ),y_(i,n_i ) ); the total number of sampling points of the strokes is n=E_(i=DAm^n__i ; a coordinate of the center of gravity of the sampling points of the strokes is (x ”,y ” ), wherein x “=l / n E_(i=l)Am^E_(j=l)A(n_i)^x_(i,j) ,y “=l / n E (i=l)Am^E (j=l)Afn i)^y (ij) ,j=l,...,n_i; the three second central moments of the sampling points 82 of the strokes 81 are I_xx=l / n E_(i=l)Am^Ejj=l)A(n_i)^(x_(i,j)-x )A2 ,I_xy=l / n E_(i=l)Am^E_(j=l)A(n_i)^(x_(i,j)-x " ) (y_(i,j)-y " ),I_yy=l / n E (i=l)Am^E (j=l)Afn i)^(y (i.j)-y ” )A2 ; an angle 0 between an x-axis and a line that minimizes the sum of the squares of the distances between each of the sample points 82 and the line is 0_l=l / 2 "arctan" ((2I_xy) / (I_xx-I_yy )) or 0_2=0_l+ir / 2; and the writing direction is the angle 0=0_k, in which k=(argmax()-r(k=l,2) I_xx cosA2 (0_k )+2I_xy cos(0_k )sin(0_k )+I_yy sinA2 (0_k )). The apparatus of claim 4, wherein the corpus is a balanced corpus or a corpus of web texts collected by a web crawler.

Images