Handwriting recognition method and apparatus, electronic device, and readable storage medium

Abstract

Embodiments of the present application provide a handwriting recognition method and device, electronic equipment and storage medium, comprising: obtaining a target number of single-character handwritten characters, calling a free writing simulation algorithm to generate a string from the target number of single-character handwritten characters, obtaining a first target stroke of the string, calling a convolutional neural network (CNN) to obtain feature information of the first target stroke, calling a bidirectional long short-term memory network (BiLSTM) to generate associated feature information for the first target stroke according to the feature information, and calling a connection time sequence classification (CTC) decoding to output a standard format character corresponding to the single-character handwritten character according to the associated feature information. That is, in the present application, the single-character handwritten character is generated into a free writing string, avoiding the limitation of a single writing method on the user, and without the need for manual design of features, the free writing recognition effect can be maximized, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of online handwriting recognition, and particularly relates to a handwriting recognition method and device, an electronic device and a readable storage medium. BACKGROUND

[0002] Handwriting input method is a very common input method on terminal devices, such as vehicle-mounted input and mobile phone input. The common handwriting input algorithm on the market requires that the input is either line writing or superimposed writing. The superimposed writing recognition needs a character hypothesis model to accurately segment and recognize a single word, and then performs path search decoding based on the single word recognition score and a language model. The line writing recognition often uses single word segmentation or a line recognition neural network algorithm.

[0003] However, the decoding process in the existing handwriting recognition method is relatively complex and can only use a single writing method, which greatly limits the user and causes poor user experience. SUMMARY

[0004] The present application provides a handwriting recognition method, device, electronic device and readable storage medium to solve the problem that the decoding process in the existing handwriting recognition method is relatively complex and can only use a single writing method, which greatly limits the user and causes poor user experience.

[0005] To solve the above technical problems, the present application is implemented as follows:

[0006] In a first aspect, the present application provides a handwriting recognition method, which comprises:

[0007] obtaining a target number of single word handwritten characters;

[0008] calling a free writing simulation algorithm to generate a string of the target number of single word handwritten characters;

[0009] obtaining a first target stroke of the string;

[0010] calling a convolutional neural network (CNN) to obtain feature information of the first target stroke;

[0011] calling a bidirectional long short-term memory network (BiLSTM) to generate associated feature information for the first target stroke according to the feature information;

[0012] calling a connection time sequence classification (CTC) decoding to output a standard format character corresponding to the single word handwritten character according to the associated feature information.

[0013] Optionally, before the obtaining a target number of single word handwritten characters, the method further comprises:

[0014] In response to a user's selection of an input method control, a target interface is entered, the target interface including a single-character handwriting mode control;

[0015] In response to a user's input to the single-character handwriting mode control, a single-character handwriting interface is displayed;

[0016] The single-character handwriting characters input by the user in the single-character handwriting interface are sent to a server for storage.

[0017] Optionally, the free writing simulation algorithm includes a line writing mode, an overlapping writing mode, and a line-overlapping mixed writing mode, and the calling of the free writing simulation algorithm to generate a string of the target number of single-character handwriting characters includes:

[0018] According to the line writing mode, the target number of single-character handwriting characters are generated into a line writing string;

[0019] or according to the overlapping writing mode, the target number of single-character handwriting characters are generated into an overlapping writing string;

[0020] or according to the line-overlapping mixed writing mode, the target number of single-character handwriting characters are generated into a line-overlapping mixed writing string.

[0021] Optionally, before the obtaining of the first target stroke of the string, the method further includes:

[0022] Obtaining a point set of the string;

[0023] According to a rule from a pen-down to a pen-up, the point set is divided into different point set groups;

[0024] According to a time sequence, a second target stroke is generated by connecting any two adjacent points in the different point set groups with a straight line.

[0025] Optionally, after the generating of the second target stroke by connecting any two adjacent points in the different point set groups with a straight line according to a time sequence, the method further includes:

[0026] Obtaining a stroke normalization height and width standard;

[0027] Obtaining an actual height and an actual width of the second target stroke;

[0028] According to the stroke normalization height and width standard, the actual height and the actual width of the second target stroke are adjusted to generate the first target stroke.

[0029] Optionally, the calling of a convolutional neural network (CNN) to obtain feature information of the first target stroke includes:

[0030] Obtaining a convolution kernel parameter of a convolutional neural network (CNN);

[0031] According to the convolution kernel parameter, the feature information of the first target stroke in different dimensions is obtained.

[0032] In a second aspect, the present application provides a handwriting recognition device, the device comprising:

[0033] A first obtaining module is configured to obtain a target number of single-character handwritten characters.

[0034] A first generating module is configured to call a free writing simulation algorithm to generate a string of characters from the target number of single-character handwritten characters.

[0035] A second obtaining module is configured to obtain a first target stroke of the string of characters.

[0036] A third obtaining module is configured to call a convolutional neural network (CNN) to obtain feature information of the first target stroke.

[0037] A second generating module is configured to call a bidirectional recurrent neural network (BiLSTM) to generate associated feature information according to the feature information.

[0038] A first output module is configured to call a connection time classification (CTC) decoding to output a standard format character corresponding to the single-character handwritten character according to the associated feature information.

[0039] In a third aspect, the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the handwriting recognition method described above when executing the program.

[0040] In a fourth aspect, the present application provides a readable storage medium, when the instructions in the storage medium are executed by the processor of an electronic device, the electronic device can execute the handwriting recognition method described above.

[0041] In the embodiments of the present application, a target number of single-character handwritten characters are obtained, a free writing simulation algorithm is called to generate a string of characters from the target number of single-character handwritten characters, a first target stroke of the string of characters is obtained, a convolutional neural network (CNN) is called to obtain feature information of the first target stroke, a bidirectional long short-term memory network (BiLSTM) is called to generate associated feature information for the first target stroke according to the feature information, and a connection time classification (CTC) decoding is called to output a standard format character corresponding to the single-character handwritten character according to the associated feature information. That is, in the present application, the single-character handwritten characters are generated into a string of free writing characters, avoiding the limitation of a single writing method on users, and the convolutional neural network (CNN) is used for feature extraction without manual design of features. Through the simultaneous use of the parameters in the CNN and the parameters in the BiLSTM and the end-to-end training of the CTC decoding, the recognition effect of free writing can be maximized, and the user experience can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0043] Figure 1 is one of the step flow charts of a handwriting recognition method provided by the embodiments of the present application;

[0044] Figure 2 is the second step flow chart of a handwriting recognition method provided by the embodiments of the present application;

[0045] Figure 3 is the third step flow chart of a handwriting recognition method provided by the embodiments of the present application;

[0046] Figure 4 is the fourth step flow chart of a handwriting recognition method provided by the embodiments of the present application;

[0047] Figure 5 is a structural diagram of a handwriting recognition device provided by the embodiments of the present application;

[0048] Figure 6 is a structural diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the protection scope of the present application.

[0050] Figure 1 is one of the step flow charts of a handwriting recognition method provided by the embodiments of the present application, as shown in Figure 1 , the method can include:

[0051] Step 101, obtaining a target number of single handwritten characters.

[0052] The sample characters for handwriting recognition in the embodiment of the application are single-character online handwritten characters of the Chinese Internal Code Specification (GbK) collected by the company and open source. Because online free writing simulation is to be performed on these characters, a target number of single-character handwritten characters are set, wherein the target number is greater than or equal to 2, and the target number can be set to 2, 3 or 5, which is not specifically limited in the application.

[0053] In step 102, a free writing simulation algorithm is called to generate a string of the target number of single-character handwritten characters.

[0054] The free writing simulation algorithm in the embodiment of the application includes a line writing mode, an overlapping writing mode and a line-overlapping mixed writing mode. The target number of single-character handwritten characters are randomly combined to generate a string in the line writing mode, the overlapping writing mode or the line-overlapping mixed writing mode.

[0055] Specifically, the target number of single-character handwritten characters are combined to generate a string according to the free writing simulation algorithm, including:

[0056] The target number of single-character handwritten characters are combined to generate a line writing string in the line writing mode;

[0057] or the target number of single-character handwritten characters are combined to generate an overlapping writing string in the overlapping writing mode;

[0058] or the target number of single-character handwritten characters are combined to generate a line-overlapping mixed writing string in the line-overlapping mixed writing mode.

[0059] In step 103, a first target stroke of the string is obtained.

[0060] In the embodiment of the application, after the target number of single-character handwritten characters are randomly combined to simulate a string according to the free writing simulation algorithm, the strokes of each character are recognized, and then all the strokes of the characters are counted to obtain all the first target strokes included in the string.

[0061] For example, four single-character handwritten characters are “I”, “love”, “China” and “country”. The four single-character handwritten characters are simulated to generate a line-overlapping mixed writing string “I love China” according to the free writing simulation algorithm. Then, all the strokes in the string are recognized to obtain the first target strokes, that is, 7 strokes in “I”, 10 strokes in “love”, 4 strokes in “China” and 8 strokes in “country”. Therefore, 29 first target strokes of the string are obtained. In addition, each first target stroke is represented by a fixed-size image, which facilitates the extraction of stroke features by a convolutional neural network (CNN) later.

[0062] Step 104, calling a convolutional neural network CNN to obtain feature information of the first target stroke.

[0063] In the embodiment of the application, the convolutional neural network CNN is called to perform feature extraction on the first target stroke to generate feature information. The input layer of the convolutional neural network can process multi-dimensional data. Commonly, the input layer of a one-dimensional convolutional neural network receives a one-dimensional or two-dimensional array, wherein the one-dimensional array is usually a time or frequency sample; the two-dimensional array can include multiple channels; the input layer of a two-dimensional convolutional neural network receives a two-dimensional or three-dimensional array; the input layer of a three-dimensional convolutional neural network receives a four-dimensional array. In this embodiment, a multi-layer convolutional neural network is used, and the output is feature information T*N, wherein T represents the number of strokes, and N represents the dimension.

[0064] For example, the embodiment of the application uses a five-layer convolutional neural network, and obtains 29 first target strokes. The 29 first target strokes are input into the five-layer convolutional neural network, and the output result is 29*10 or 29*20, that is, the feature extraction of the first target stroke from the 10-dimensional or 20-dimensional perspective is output.

[0065] It should be noted that the convolutional network includes a plurality of convolutional layers and pooling layers. The pooling layer is used to reduce the number of neurons, and the convolutional layer can extract different features. The training of the CNN focuses on training various convolution kernels to extract features. Therefore, the convolutional network of the embodiment of the application needs to set appropriate convolutional layer parameters and pooling layer parameters, obtain a plurality of different convolution kernel parameters through these parameters, and obtain feature information of the first target stroke in different dimensions according to the convolution kernel parameters. The specific steps include:

[0066] Obtaining the convolution kernel parameters of the convolutional neural network CNN.

[0067] Obtaining the feature information of the first target stroke in different dimensions according to the convolution kernel parameters.

[0068] Step 105, calling a bidirectional long short-term memory network BiLSTM to generate associated feature information for the first target stroke according to the feature information.

[0069] After the features of the first target stroke are extracted in the embodiment of the application, a bidirectional long short-term memory network BiLSTM is called to associate the strokes in context, and the associated feature information of the first target stroke is generated. The bidirectional long short-term memory network has a connection not only with the past state but also with the future state, so the BiLSTM can be regarded as two layers of neural networks, the first layer is input from the left as the beginning of the series, which can be understood as input from the beginning of the sentence in text processing, and the second layer is input from the right as the beginning of the series, which can be understood as input from the last word of the sentence in text processing, and the same processing as the first layer is performed in reverse. Finally, the two results are processed, and for the embodiment of the application, one result is obtained by inputting the strokes in the normal order of time for one character, and another result is obtained by inputting the strokes in the reverse order of time, and the two results are combined to generate the associated feature information of the first target stroke.

[0070] For example, the stroke order of the character "middle" obtained according to the associated feature information of the first target stroke in the embodiment of the application is "one", "one", According to the steps, the correct order of all strokes of a character can be obtained by associating the strokes in context, that is, the associated feature information of the first target stroke is obtained.

[0071] In step 106, CTC decoding is called to output the standard format character corresponding to the single handwritten character according to the associated feature information.

[0072] In the embodiment of the application, after the first target stroke of the string is associated in context, CTC decoding is called to combine the strokes in a suitable position and manner, and then output, to obtain the standard format character corresponding to the single handwritten character. In addition, because the associated feature information after feature extraction has repeated stroke information, CTC decoding is also required to obtain a unique and correct standard format character. CTC decoding increases a many-to-one space mapping between the output sequence and the final label, and defines the many-to-one condition on this basis.

[0073] For example, for the character "middle", the stroke order obtained according to the associated feature information of the first target stroke is "one", "one", "one", "one", but because the stroke "one" is repeated, the obtained character is not "middle". In order to solve this problem, CTC decoding introduces a placeholder, denoted as ∈ or -. It represents an occupation and does not correspond to any character, so it is deleted in the final output. At this time, the alignment mode is "one", "one", "∈", "∈", "one", "∈", "one", so that the output result corresponds to "Chinese".

[0074] Therefore, the embodiment of the application obtains the associated feature information of the first target stroke, and then calls the CTC decoding to perform the end-to-end supervised training to output the standard format character corresponding to the single handwritten character.

[0075] In the embodiment of the application, the target number of single handwritten characters is obtained, the free writing simulation algorithm is called to generate a string of the target number of single handwritten characters, the first target stroke of the string is obtained, the convolutional neural network (CNN) is called to obtain the feature information of the first target stroke, the bidirectional long short-term memory network (BiLSTM) is called to generate the associated feature information for the first target stroke according to the feature information, and the CTC decoding is called to output the standard format character corresponding to the single handwritten character according to the associated feature information. That is, in the application, the single handwritten character is generated into a free writing string, avoiding the limitation of a single writing method on the user. At the same time, the feature extraction is performed by the convolutional neural network, without the need for manual design of features. The end-to-end training by the CTC decoding using the parameters in the CNN and the parameters in the BiLSTM can maximize the recognition effect of free writing and improve the user experience.

[0076] Figure 2 is a step flowchart of a handwriting recognition method provided by the embodiment of the application. The handwriting recognition method disclosed in the embodiment has basically the same steps as those shown in Figure 1 , and the difference is that before step 101, it can further include:

[0077] Step 201: in response to the user's selection of the input method control, enter the target interface, and the target interface includes a single handwritten mode control.

[0078] In the embodiment of the application, some single handwritten characters input by the user are collected in advance before the free writing string is generated. Therefore, the user will open the input method to enter the target interface. The user can enter the target interface by clicking the input method control or inputting a voice instruction, which is not specifically limited in the application.

[0079] Because the handwritten single character is obtained, the selected input method includes a single handwritten mode control. In addition, the target interface of the input method also has other forms of input format controls, such as pinyin nine-key input, five-stroke input, pinyin twenty-six-key input, and various input format controls, which are not specifically limited in the application.

[0080] Step 202: in response to the user's input to the single handwritten mode control, display a single handwritten interface.

[0081] The user selects and clicks a single-character handwriting mode control or a voice-selected single-character handwriting mode in the target interface, and a single-character handwriting interface is displayed, wherein the single-character handwriting interface can be a newly popped-up interface or an interface divided in the target interface.

[0082] In step 203, the single-character handwriting character input by the user in the single-character handwriting interface is sent to the server for storage.

[0083] In the embodiment of the application, the single-character handwriting character input by the user in the single-character handwriting interface is sent to the server for storage, which is also convenient for obtaining a plurality of different single-character handwriting characters from the server for free writing simulation.

[0084] In the embodiment of the application, the target number of single-character handwriting characters are obtained, a free writing simulation algorithm is called to generate a string from the target number of single-character handwriting characters, a first target stroke of the string is obtained, a convolutional neural network (CNN) is called to obtain feature information of the first target stroke, a bidirectional long short-term memory network (BiLSTM) is called to generate associated feature information for the first target stroke according to the feature information, and a connectionist temporal classification (CTC) decoding is called to output a standard format character corresponding to the single-character handwriting character according to the associated feature information. That is, in the application, the single-character handwriting character is generated into a free writing string, which avoids the limitation of a single writing method on the user, and the feature extraction is performed through the convolutional neural network, without manual design of the feature. The end-to-end training through the parameters in the CNN and the parameters in the BiLSTM and the CTC decoding can maximize the recognition effect of the free writing and improve the user experience.

[0085] Figure 3 The embodiment of the application provides a step flowchart of a handwriting recognition method, and the handwriting recognition method disclosed in the embodiment has basically the same steps as those shown in the Figure 1 The embodiment of the application provides a step flowchart of a handwriting recognition method, and the handwriting recognition method disclosed in the embodiment has basically the same steps as those shown in the

[0086] In step 301, a point set of a string is obtained.

[0087] In the embodiment of the application, the string generated from the single-character handwriting character is stored in the form of a point set. For example, a stroke “|” is composed of 30 point sets. Since the string includes a plurality of strokes, each stroke is composed of point sets, and therefore, in order to divide the strokes, the point set of the string needs to be obtained first.

[0088] In step 302, the point set is divided into different point set groups according to the rule from the beginning of writing to the end of writing.

[0089] In the embodiment of the present application, the pen-down to pen-up is regarded as a stroke when inputting online, so the point set is divided into different point set groups according to the rule from pen-down to pen-up, wherein each point set group represents a stroke, so that the strokes can be divided.

[0090] In step 303, the second target stroke is generated by connecting any two adjacent points in different point set groups with a straight line in time sequence.

[0091] In the embodiment of the present application, after the point set is divided according to the stroke, because the final recognition is a character, not a point set, so the second target stroke is generated by connecting any two adjacent points in the point set group with a straight line in time sequence. It should be noted that because the point set is connected in time sequence, the time information of the point set is also saved when the point set is saved.

[0092] In the embodiment of the present application, the target number of single-character handwritten characters is obtained, the free writing simulation algorithm is called to generate a string of the target number of single-character handwritten characters, the first target stroke of the string is obtained, the convolutional neural network CNN is called to obtain the feature information of the first target stroke, the bidirectional long short-term memory network BiLSTM is called to generate the associated feature information for the first target stroke according to the feature information, and the CTC decoding is called to output the standard format character corresponding to the single-character handwritten character according to the associated feature information. That is, in the present application, the single-character handwritten character is generated into a free writing string, which avoids the limitation of a single writing method on the user, and the feature extraction is performed through the convolutional neural network, without manual design of the feature. Through the end-to-end training of the parameters in the CNN and the parameters in the BiLSTM and the CTC decoding, the free writing recognition effect can be maximized, and the user experience can be improved.

[0093] Figure 4 The third step flow chart of the handwriting recognition method provided in the embodiment of the present application is basically the same as the handwriting recognition method disclosed in the embodiment of the present application, and the difference is that after step 303, the method can further include: Figure 3

[0094] In step 401, the height and width standards of stroke normalization are obtained.

[0095] In the embodiment of the present application, after the strokes are divided according to the point set, the strokes need to be linearly normalized, so the height and width standards of stroke normalization are obtained, for example, the height of the stroke is normalized to 128, and the width is normalized to 64. The corresponding height and width can also be designed according to the writing display interface, which is not limited in the present application.

[0096] In step 402, the actual height and actual width of the second target stroke are obtained. ​

[0097] The embodiment of the application sets up a rectangular coordinate system according to the first point of the second target stroke as the origin, judges the basic stroke according to the trajectory formed by the force direction and time of the writing head when writing, collects the trajectory formed by the basic stroke, and when the basic stroke is vertical, the trajectory of the force and time is in the negative direction of the Y axis, when the basic stroke is horizontal, the trajectory of the force and time is in the positive direction of the X axis, when the basic stroke is a dot, the trajectory of the force and time is in the fourth quadrant of the coordinate system and the length of the trajectory is shorter than that of the dot, when the basic stroke is a fold, the trajectory of the force of the horizontal fold first appears in the positive direction of the X axis, then quickly turns to the negative direction of the Y axis after a period of time, and the trajectory of the force of the vertical fold first appears in the third quadrant, then quickly turns to the positive direction of the X axis after a period of time, the trajectories of different second target strokes are determined in the above manner, and the actual height and actual width of the second target stroke are determined according to the coordinate system.

[0098] In step 403, the actual height and actual width of the second target stroke are adjusted according to the height and width standard of stroke normalization to generate a first target stroke.

[0099] In the embodiment of the application, the actual height and actual width of the second target stroke are adjusted according to the height and width standard of stroke normalization, and the first target stroke is generated after all the second target strokes are adjusted according to the normalization standard.

[0100] In the embodiment of the application, the target number of single handwritten characters is obtained, the free writing simulation algorithm is called to generate a string of characters from the target number of single handwritten characters, the first target stroke of the string is obtained, the convolutional neural network (CNN) is called to obtain the feature information of the first target stroke, the bidirectional long short-term memory network (BiLSTM) is called to generate the associated feature information of the first target stroke according to the feature information, and the CTC decoding is called to output the standard format character corresponding to the single handwritten character according to the associated feature information. That is, in the application, the single handwritten character is generated into a free writing string, avoiding the limitation of a single writing method on the user, and the feature extraction is performed through the convolutional neural network, without manual design of the feature. Through the end-to-end training of the parameters in the CNN and the parameters in the BiLSTM and the CTC decoding, the free writing recognition effect can be maximized, and the user experience can be improved.

[0101] Figure 5 is a structural diagram of a handwriting recognition device provided by the embodiment of the application, which can include:

[0102] The first acquisition module 501 is configured to acquire a target number of single handwritten characters.

[0103] The first generation module 502 is configured to call a free writing simulation algorithm to generate a target number of single handwritten characters into a string.

[0104] The second acquisition module 503 is configured to acquire a first target stroke of the string.

[0105] The third acquisition module 504 is configured to call a convolutional neural network (CNN) to acquire feature information of the first target stroke.

[0106] The second generation module 505 is configured to call a bidirectional recurrent neural network (BiLSTM) to generate associated feature information according to the feature information.

[0107] The first output module 506 is configured to call a connection time sequence classification (CTC) decoding to output a standard format character corresponding to the single handwritten character according to the associated feature information.

[0108] Optionally, the handwriting recognition apparatus further comprises:

[0109] The selection module is configured to enter a target interface in response to a user selection of an input method control, the target interface comprising a single handwritten mode control.

[0110] The display module is configured to display a single handwritten interface in response to a user input on the single handwritten mode control.

[0111] The sending module is configured to send single handwritten characters input by the user on the single handwritten interface to a server for storage.

[0112] Optionally, the free writing simulation algorithm comprises a line writing mode, an overlapping writing mode, and a line-overlapping mixed writing mode, and the first generation module further comprises:

[0113] The first generation sub-module is configured to generate a target number of single handwritten characters into a line writing string according to the line writing mode.

[0114] or generate a target number of single handwritten characters into an overlapping writing string according to the overlapping writing mode.

[0115] or generate a target number of single handwritten characters into a line-overlapping mixed writing string according to the line-overlapping mixed writing mode.

[0116] Optionally, the handwriting recognition apparatus further comprises:

[0117] The fourth acquisition module is configured to acquire a point set of the string.

[0118] The division module is configured to divide the point set into different point set groups according to a rule from a pen-down to a pen-up.

[0119] The third generating module is configured to generate the second target stroke by connecting any two adjacent points in the different point set groups in a straight line in chronological order.

[0120] The fifth obtaining module is configured to obtain a normalized height and width standard of the stroke.

[0121] The sixth obtaining module is configured to obtain an actual height and an actual width of the second target stroke.

[0122] The fourth generating module is configured to generate the first target stroke by adjusting the actual height and the actual width of the second target stroke according to the normalized height and width standard of the stroke.

[0123] Optionally, the third obtaining module further comprises:

[0124] The first obtaining submodule is configured to obtain a convolution kernel parameter of a convolutional neural network (CNN).

[0125] The second obtaining submodule is configured to obtain feature information of the first target stroke in different dimensions according to the convolution kernel parameter.

[0126] In the embodiment of the present application, a target number of single-character handwritten characters are obtained, a free writing simulation algorithm is called to generate a string of the target number of single-character handwritten characters, a first target stroke of the string is obtained, a convolutional neural network (CNN) is called to obtain feature information of the first target stroke, a bidirectional long short-term memory network (BiLSTM) is called to generate associated feature information for the first target stroke according to the feature information, and a connectionist temporal classification (CTC) decoding is called to output a standard format character corresponding to the single-character handwritten character according to the associated feature information. That is, in the present application, the single-character handwritten characters are generated into a string of free writing, which avoids the limitation of a single writing method on users, and the feature extraction is performed by the convolutional neural network, without the need for manual design of features. The parameters in the CNN and the parameters in the BiLSTM are used simultaneously, and the CTC decoding is used for end-to-end training, which can maximize the recognition effect of free writing and improve the user experience.

[0127] The present application also provides an electronic device, Figure 6 is a structural block diagram of an electronic device provided by the embodiment of the present application, referring to Figure 6 , comprising a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 complete mutual communication through the communication bus 604,

[0128] The memory 603 is used to store a computer program.

[0129] The processor 601 is used to execute the program stored in the memory 603, and the following steps are implemented:

[0130] Obtaining a target number of single-word handwritten characters;

[0131] Calling a free writing simulation algorithm to generate a string of the target number of single-word handwritten characters;

[0132] Obtaining a first target stroke of the string;

[0133] Calling a convolutional neural network (CNN) to obtain feature information of the first target stroke;

[0134] Calling a bidirectional long short-term memory network (BiLSTM) to generate associated feature information for the first target stroke according to the feature information;

[0135] Calling a connection time sequence classification (CTC) decoding to output a standard format character corresponding to the single-word handwritten character according to the associated feature information.

[0136] The application further provides a readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the handwriting recognition method of the foregoing embodiments.

[0137] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts refer to the part of the method embodiment.

[0138] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Structural requirements of a system required to construct such a system are apparent from the description above. Moreover, the application is not directed to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the present application described herein, and that the descriptions of specific languages are provided for disclosure of the best mode of the application.

[0139] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the application can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not shown in detail in order not to obscure the understanding of the specification.

[0140] Similarly, it is to be understood that the embodiments of the present application can be positioned and described in a number of different orientations, and the terminology or description can be chosen for purposes of convenience only. It is therefore intended that the application include all such modifications and alterations in the depiction of the essential aspects of the preferred embodiments of the application that were required over and above the normal variations inherent in any significant, modular, reusable hardware or software components. Additionally, the term "data bus" as used throughout this detailed description shall be interpreted to include any type of bus structure or bus system, including a memory bus, address bus, peripheral bus, or other busses that are known to those skilled in the art.

[0141] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or steps of any method so disclosed can be made unless specifically stated otherwise. Each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features serving the same, equivalent or similar purpose unless specifically stated otherwise.

[0142] Embodiments of the various components of the application can be implemented in hardware, or as software modules running in one or more processors, or some combination of both. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some or all of the components in the sequencing apparatus according to the present application. The present application can also be implemented as a program for executing the part or all of the methods described herein on a device or apparatus. Such a program can be stored on a computer readable medium or can have one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier medium, or in any other form.

[0143] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a comma. The use of the term 'about' followed by a value and / or a term 'approximately' preceding a value means that the value can vary from the stated value by 10%. The use of any of the following terms in the claims is neither meant to limit the scope nor to introduce a non-combination limitation. The terms 'comprise', 'include', and 'contain' are not used in their exclusive sense. The use of the term 'first','second', and 'third' does not connote any order, quantity, creation or importance, but rather are used to denote one element from another. The use of these terms is interchangeable under appropriate circumstances.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0145] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall fall within the scope of the present application.

[0146] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall fall within the scope of the present application.

[0147] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a comma. The use of the term 'about' followed by a value and / or a term 'approximately' preceding a value means that the value can vary from the stated value by 10%. The use of any of the following terms in the claims is neither meant to limit the scope nor to introduce a non-combination limitation. The terms 'comprise', 'include', and 'contain' are not used in their exclusive sense. The use of the term 'first','second', and 'third' does not connote any order, quantity, creation or importance, but rather are used to denote one element from another. The use of these terms is interchangeable under appropriate circumstances.

Claims

1. A handwriting recognition method, characterized in that, The method includes: Get the target number of single-character handwritten characters; The free-write simulation algorithm is invoked to generate a string from the target number of handwritten single characters; Obtain the first target stroke of the string; The convolutional neural network (CNN) is invoked to obtain the feature information of the first target stroke. The Bidirectional Long Short-Term Memory (BiLSTM) network is invoked to generate associated feature information for the first target stroke based on the aforementioned feature information; The connection timing classification (CTC) decoder is invoked to output the standard format character corresponding to the single handwritten character based on the associated feature information. The freehand writing simulation algorithm includes: line writing mode, overlapping writing mode, and line-overlapping mixed writing mode. The step of calling the freehand writing simulation algorithm to generate a string from the target number of handwritten single characters includes: Generate a line-written string from the target number of single-character handwritten characters according to the line-writing pattern; Alternatively, the target number of handwritten single characters can be generated into an overprinted string according to the overprinting pattern; Alternatively, the target number of handwritten single characters can be generated into a line-overlapping mixed string according to the line-overlapping mixed writing mode; Before obtaining the first target stroke of the string, the method further includes: Obtain the dot set of the string; The point set is divided into different point set sets according to the rule from the start of the stroke to the end of the stroke; The second target stroke is generated by connecting any two adjacent points in the different point sets in chronological order with a straight line. After generating the second target stroke by connecting any two adjacent points in the different point sets in chronological order with straight lines, the method further includes: Obtain the standard height and width of strokes after normalization; Obtain the actual height and actual width of the second target stroke; The first target stroke is generated by adjusting the actual height and actual width of the second target stroke according to the stroke normalization height and width standard. The process of obtaining the actual height and actual width of the second target brush includes: Based on the first point of the second target stroke as the origin, a rectangular coordinate system is constructed. The basic stroke is determined by the trajectory formed by the direction of the force and time of the writing head in the rectangular coordinate system. The acquisition module collects the formed trajectory. When the basic stroke is vertical, the trajectory of force and time is in the negative direction of the Y-axis; when the basic stroke is horizontal, the trajectory of force and time is in the positive direction of the X-axis; when the basic stroke is a downward stroke, the trajectory of force and time is in the fourth quadrant of the coordinate system; when the basic stroke is a left-falling stroke, the trajectory of force and time is in the third quadrant of the coordinate system; when the basic stroke is a dot, the trajectory of force and time is in the fourth quadrant and the length of the trajectory is shorter than that of the downward stroke; when the basic stroke is a turning stroke, it is divided into horizontal turning and left-falling turning. The trajectory of the force of the horizontal turning stroke first appears in the positive region of the X-axis as time goes by, and after a period of time, it quickly turns to the negative region of the Y-axis and then after another period of time. The trajectory of the force of the left-falling turning stroke first appears in the third quadrant as time goes by, and after a period of time, it quickly turns to the positive region of the X-axis. Different second target stroke trajectories are determined in the above manner, and the actual height and actual width of the second target stroke are determined according to the coordinate system.

2. The method according to claim 1, characterized in that, Before obtaining the target number of handwritten single characters, the method further includes: In response to the user's selection of the input method control, the target interface is entered, the target interface including a single-character handwriting mode control; In response to user input on the single-character handwriting mode control, the single-character handwriting interface is displayed; The single-character handwritten character entered by the user on the single-character handwriting interface is sent to the server for storage.

3. The method according to claim 1, characterized in that, The step of calling a convolutional neural network (CNN) to obtain the feature information of the first target stroke includes: Obtain the convolution kernel parameters of a convolutional neural network (CNN); The feature information of the first target stroke in different dimensions is obtained based on the convolution kernel parameters.

4. A handwriting recognition device, characterized in that, The device includes: The first acquisition module is used to acquire a target number of single-character handwritten characters; The first generation module is used to call the free-write simulation algorithm to generate a string from the target number of single-character handwritten characters; The second acquisition module is used to acquire the first target stroke of the string; The third acquisition module is used to call a convolutional neural network (CNN) to acquire the feature information of the first target stroke. The second generation module is used to call the bidirectional recurrent neural network BiLSTM to generate associated feature information based on the feature information; The first output module is used to call the Connection Time Classification (CTC) decoder to output the standard format character corresponding to the single handwritten character based on the associated feature information. Optionally, the free write simulation algorithm includes: line write mode, overlay write mode, and line-overlay mixed write mode, and the first generation module further includes: The first generation submodule is used to generate a line-written string from a target number of single-character handwritten characters according to the line-writing pattern; Alternatively, generate an overprinted string from the target number of handwritten single characters according to the overprinting pattern; Alternatively, generate a line-over-line compositing string from the target number of handwritten single characters according to the line-over-line compositing pattern; The fourth acquisition module is used to acquire the dot set of the string; The partitioning module is used to divide the point set into different point sets according to the rules from the start of the stroke to the end of the stroke. The third generation module is used to connect any two adjacent points in different point sets with straight lines in chronological order to generate the second target stroke. The fifth acquisition module is used to obtain the height and width standards of stroke normalization; The sixth acquisition module is used to acquire the actual height and actual width of the second target stroke; The fourth generation module is used to adjust the actual height and width of the second target stroke according to the height and width standard of stroke normalization to generate the first target stroke; The process of obtaining the actual height and actual width of the second target brush includes: Based on the first point of the second target stroke as the origin, a rectangular coordinate system is constructed. The basic stroke is determined by the trajectory formed by the direction of the force and time of the writing head in the rectangular coordinate system. The acquisition module collects the formed trajectory. When the basic stroke is vertical, the trajectory of force and time is in the negative direction of the Y-axis; when the basic stroke is horizontal, the trajectory of force and time is in the positive direction of the X-axis; when the basic stroke is a downward stroke, the trajectory of force and time is in the fourth quadrant of the coordinate system; when the basic stroke is a left-falling stroke, the trajectory of force and time is in the third quadrant of the coordinate system; when the basic stroke is a dot, the trajectory of force and time is in the fourth quadrant and the length of the trajectory is shorter than that of the downward stroke; when the basic stroke is a turning stroke, it is divided into horizontal turning and left-falling turning. The trajectory of the force of the horizontal turning stroke first appears in the positive region of the X-axis as time goes by, and after a period of time, it quickly turns to the negative region of the Y-axis and then after another period of time. The trajectory of the force of the left-falling turning stroke first appears in the third quadrant as time goes by, and after a period of time, it quickly turns to the positive region of the X-axis. Different second target stroke trajectories are determined in the above manner, and the actual height and actual width of the second target stroke are determined according to the coordinate system.

5. The apparatus according to claim 4, characterized in that, The handwriting recognition device also includes: The selection module is used to respond to the user's selection of the input method control and enter the target interface, which includes a single-character handwriting mode control. The display module is used to display the single-character handwriting interface in response to user input on the single-character handwriting mode control; The sending module is used to send the single-character handwritten character entered by the user on the single-character handwriting interface to the server for storage.

6. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the program, implements the handwriting recognition method as described in any one of claims 1-3.

7. A readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform one or more of the handwriting recognition methods described in claims 1-3.

Images