Handwritten text recognition methods, apparatuses, and systems, handwritten text search methods and systems, and computer-readable storage media

Abstract

The present disclosure relates to a handwritten text recognition method, device and system, a handwritten text search method and system, and a computer readable storage medium. The handwritten text recognition method comprises: obtaining an information sequence of a handwritten text comprising a plurality of trajectory points, the information of each trajectory point comprising an abscissa, a writing time and a writing state value of the trajectory point; dividing the plurality of trajectory points into a plurality of strokes according to the writing state value of each trajectory point, wherein the writing state value comprises a first value representing lifting of a stroke and a second value representing landing of a stroke; calculating a first segmentation threshold value of the handwritten text; determining a first text segmentation point according to a comparison result of a differential absolute value of the abscissa of a starting point trajectory point of a stroke and an ending point trajectory point of a previous stroke with the first segmentation threshold value; and performing text segmentation according to the first text segmentation point to obtain a text segmentation result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer technology, and in particular, to a handwriting text recognition method, device and system, a handwriting text search method and system, and a computer readable storage medium. BACKGROUND

[0002] Compared with traditional keyboard input, handwriting input is widely used due to its convenience and efficiency. Handwriting recognition provides convenience for natural human-computer interaction. With the development of technology, touch screens provide a better handwriting input mode for users, allowing users to input multiple lines of text online.

[0003] To implement handwriting recognition of multi-line handwriting input, the multi-line text needs to be first segmented into single-line text, and the obtained text line sequence is then input to a system capable of single-line continuous handwriting recognition for handwriting recognition. Therefore, the accuracy of multi-line text segmentation has an important influence on the performance of the handwriting recognition system.

[0004] Related text line segmentation techniques are mainly based on analysis of the horizontal and vertical axis projection histogram of continuous handwriting, and the accuracy of segmentation is not high. SUMMARY

[0005] According to some embodiments of the present disclosure, a handwriting text recognition method is provided, comprising:

[0006] obtaining an information sequence of a handwriting text comprising a plurality of trajectory points, the information of each trajectory point comprising a horizontal coordinate, a writing time and a writing state value of the trajectory point;

[0007] dividing the plurality of trajectory points into a plurality of strokes according to the writing state value of each trajectory point, wherein the writing state value comprises a first value representing lifting of a stroke and a second value representing landing of a stroke;

[0008] calculating a first segmentation threshold of the handwriting text;

[0009] determining a first text segmentation point according to a comparison result of a difference absolute value of horizontal coordinates of a starting point trajectory point of a stroke and an ending point trajectory point of a previous stroke with the first segmentation threshold;

[0010] performing text segmentation according to the first text segmentation point to obtain a text segmentation result.

[0011] In some embodiments, determining the first text segmentation point comprises: in a case where the difference absolute value of the horizontal coordinates of the starting point trajectory point of the stroke and the ending point trajectory point of the previous stroke is greater than the first segmentation threshold, determining the starting point trajectory point as the first text segmentation point.

[0012] In some embodiments, the first segmentation threshold comprises a first sub-segmentation threshold, and the calculating the first segmentation threshold of the handwritten text comprises: calculating the first sub-segmentation threshold of the handwritten text according to statistical information of a difference absolute value of horizontal coordinates of a start point of each stroke in the handwritten text and an end point of a previous stroke.

[0013] In some embodiments, the statistical information comprises a mean value and a variance, and the calculating the first segmentation threshold of the handwritten text comprises: calculating the first sub-segmentation threshold according to a sum of the mean value and the variance.

[0014] In some embodiments, the second segmentation threshold further comprises a second sub-segmentation threshold, and the calculating the segmentation threshold of the handwritten text comprises: calculating the second sub-segmentation threshold according to a difference between a maximum value and a minimum value of the horizontal coordinates of the plurality of trajectory points.

[0015] In some embodiments, in a case where the variance is greater than or equal to the mean value or the first segmentation threshold is greater than or equal to the second segmentation threshold, it is determined whether a difference absolute value of horizontal coordinates of a start point of a stroke and an end point of a previous stroke is greater than the first segmentation threshold.

[0016] In some embodiments, in a case where the difference absolute value is greater than the first segmentation threshold, the start point of the stroke is determined as a first text segmentation point comprises: in a case where the difference absolute value is greater than the first sub-segmentation threshold or the second sub-segmentation threshold, the start point of the stroke is determined as the first text segmentation point.

[0017] In some embodiments, the handwritten text recognition method further comprises:

[0018] Before determining the first text segmentation point, a second text segmentation point is determined according to a comparison result of a difference between writing times corresponding to the plurality of strokes and a second segmentation threshold;

[0019] According to the second text segmentation point, the plurality of strokes are divided into at least one text segmentation unit, and a difference between writing times of strokes in each text segmentation unit is less than or equal to the second segmentation threshold.

[0020] In some embodiments, the information of each trajectory point further comprises a vertical coordinate of the trajectory point, and the handwritten text recognition method further comprises:

[0021] Before determining the first text segmentation point, a third text segmentation point is determined according to a comparison result of a distance between a start point of a stroke and an end point of a previous stroke and a third segmentation threshold;

[0022] According to the third text segmentation point, strokes in each text segmentation unit obtained according to the second text segmentation point are segmented to obtain a plurality of text segmentation sub-units, wherein a start point track point of a stroke in each text segmentation sub-unit and an end point track point of a previous stroke are less than or equal to the third segmentation threshold.

[0023] In some embodiments, the third segmentation threshold is determined according to an average length of the plurality of strokes.

[0024] In some embodiments, the first segmentation threshold is determined according to an average length of strokes in each text segmentation sub-unit.

[0025] In some embodiments, the text segmentation according to the first text segmentation point obtains a text segmentation result, which includes:

[0026] Based on a comparison result of an overlapping degree between a range of vertical coordinates of track points included in each text segmentation unit obtained according to the first text segmentation point and a merging threshold, the text segmentation units are merged to obtain a text segmentation result.

[0027] In some embodiments, the handwriting text recognition method further includes:

[0028] According to the information sequence of the handwriting text including a plurality of track points, a handwriting text image is formed;

[0029] An offline text detection algorithm is used to perform text detection on the handwriting text image to obtain at least one text detection box;

[0030] Each track point belonging to a text detection box is identified;

[0031] According to the identification result, the text segmentation result is corrected.

[0032] In some embodiments, according to the identification result, the text segmentation result is corrected, which includes:

[0033] Track points not belonging to any text detection box are deleted;

[0034] For track points belonging to a text detection box, it is determined whether two adjacent track points belong to different text detection boxes;

[0035] In the case where two adjacent track points belong to different text detection boxes, the two adjacent track points are segmented to obtain two text segmentation sub-units.

[0036] In some embodiments, the information of each track point further includes a vertical coordinate of the track point, and according to the information sequence of the handwriting text including a plurality of track points, a handwriting text image is formed, which includes:

[0037] The maximum and minimum values ​​of the x-coordinates Xmax and Xmin, and the maximum and minimum values ​​of the y-coordinates Ymax and Ymin of the multiple trajectory points are statistically analyzed to obtain the x-coordinate range Xrange = Xmax - Xmin and the y-coordinate range Yrange = Ymax - Ymin.

[0038] Calculate the scaling factors RX = Iwidth / Xrange and RY = Iheight / Yrange based on the specified image dimensions Iwidth and Iheight, the horizontal coordinate range Xrange and the vertical coordinate range Yrange;

[0039] The x-coordinate and y-coordinate of each trajectory point are scaled according to the scaling factors RX and RY respectively, resulting in multiple trajectory points with scaled coordinates;

[0040] The handwritten text image is formed by connecting multiple trajectory points after coordinate scaling, wherein the endpoint trajectory point of one stroke is not connected to the starting trajectory point of the next stroke.

[0041] In some embodiments, the text detection box identifying the location of each trajectory point includes:

[0042] Text regions enclosed by different text detection boxes are identified to generate text detection label images. In the text detection label images, image regions that do not belong to any text region have a first pixel value, text regions enclosed by the same text detection box have the same second pixel value, and text regions enclosed by different text detection boxes have different second pixel values.

[0043] Based on the pixel value of each trajectory point at its corresponding position in the text detection and labeling image, the text detection box to which each trajectory point belongs is identified.

[0044] In some embodiments, the handwritten text recognition method further includes: for a trajectory point in each stroke, if the difference between the slope of the line connecting the trajectory point and its predecessor and successor trajectory points is less than or equal to a change threshold, the trajectory point is deleted.

[0045] In some embodiments, the handwritten text recognition method further includes: normalizing the ordinate of the trajectory points included in each text segmentation unit, and scaling the abscissa proportionally to the ordinate.

[0046] In some embodiments, the handwritten text recognition method further includes: using the minimum bounding rectangle algorithm to obtain the angle of each text segmentation unit relative to the horizontal direction;

[0047] Based on the angle of each text segmentation unit, the trajectory points included in each text segmentation unit are rotated and transformed.

[0048] In some embodiments, the handwriting text recognition method further comprises: recognizing the text segmentation result by using a text recognition model of bidirectional long short-term memory (BiLSTM) to obtain a text recognition result.

[0049] According to some other embodiments of the present disclosure, a handwriting text search method is provided, comprising:

[0050] obtaining an information sequence of handwriting text to be searched, the information sequence comprising a plurality of trajectory points;

[0051] dividing the plurality of trajectory points into a plurality of strokes;

[0052] recognizing the handwriting text to be searched by using a stroke-based handwriting text recognition model to obtain a text recognition result;

[0053] in response to a search command input by a user, performing a corresponding search in the obtained text recognition result.

[0054] In some embodiments, recognizing the handwriting text to be searched to obtain a text recognition result comprises: recognizing the handwriting text to be searched according to the handwriting text recognition method of any one of the preceding embodiments to obtain a text recognition result.

[0055] According to some other embodiments of the present disclosure, a handwriting text recognition apparatus is provided, comprising:

[0056] an obtaining unit configured to obtain an information sequence of handwriting text, the information sequence comprising a plurality of trajectory points, and the information of each trajectory point comprising a horizontal coordinate, a writing time and a writing state value of the trajectory point;

[0057] a dividing unit configured to divide the plurality of trajectory points into a plurality of strokes according to the writing state value of each trajectory point, wherein the writing state value comprises a first value representing a pen-up and a second value representing a pen-down of a stroke respectively;

[0058] a calculating unit configured to calculate a first segmentation threshold value of the handwriting text;

[0059] a determining unit configured to determine a first text segmentation point according to a comparison result of a differential absolute value of horizontal coordinates of a start trajectory point of a stroke and an end trajectory point of a previous stroke with the first segmentation threshold value; and

[0060] a segmenting unit configured to perform text segmentation according to the first text segmentation point to obtain a text segmentation result.

[0061] According to some other embodiments of the present disclosure, an electronic device is provided, comprising:

[0062] a memory; and

[0063] a processor coupled to the memory, the processor configured to perform one or more steps of the handwriting text recognition method or the handwriting text search method according to any one of the preceding embodiments based on instructions stored in the memory device.

[0064] According to still some embodiments of the present disclosure, there is provided a handwriting text recognition system, comprising:

[0065] a touch screen configured to generate a sequence of information comprising a plurality of trajectory points when a user writes characters on the touch screen;

[0066] the handwriting text recognition device according to any one of the preceding embodiments.

[0067] In some embodiments, the handwriting text recognition system further comprises a terminal configured to acquire the sequence of information comprising a plurality of trajectory points generated when a user writes characters on the touch screen and send to the handwriting text recognition device.

[0068] According to still some embodiments of the present disclosure, there is provided a handwriting text search system, comprising:

[0069] the handwriting text recognition system according to any one of the preceding embodiments, configured to recognize handwriting text to be searched and obtain a text recognition result;

[0070] a search engine configured to perform corresponding search in the obtained text recognition result in response to a search command input by a user.

[0071] According to other embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements one or more steps of the handwriting text recognition method or the handwriting text search method according to any one of the preceding embodiments.

[0072] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0073] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0074] The present disclosure can be understood more readily by reference to the following detailed description of exemplary embodiments and the attached drawings, of which:

[0075] Figure 1 is a flowchart illustrating a handwriting text recognition method according to an embodiment of the present disclosure;

[0076] Figure 1A is a flowchart illustrating a handwriting text segmentation method according to an embodiment of the present disclosure;

[0077] Figure 2 FIG. 18 is a flowchart illustrating a handwriting text recognition method according to another embodiment of the present disclosure;

[0078] Figure 2A FIG. 19 is a flowchart illustrating forming a handwriting text image according to an embodiment of the present disclosure;

[0079] Figure 2B FIG. 20 is a flowchart illustrating identifying a text detection box to which each trajectory point belongs according to an embodiment of the present disclosure;

[0080] Figure 2C FIG. 21 is a flowchart illustrating text segmentation according to a result of identification according to an embodiment of the present disclosure;

[0081] Figure 2D FIG. 22 is a schematic diagram illustrating a slanted text line according to an embodiment of the present disclosure;

[0082] Figure 3A FIG. 23 is a block diagram illustrating a handwriting text recognition apparatus according to an embodiment of the present disclosure;

[0083] Figure 3B FIG. 24 is a block diagram illustrating a handwriting text recognition apparatus according to another embodiment of the present disclosure;

[0084] Figure 4 FIG. 25 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure;

[0085] Figure 5A FIG. 26 is a block diagram illustrating a handwriting text recognition system according to an embodiment of the present disclosure;

[0086] Figure 5B FIG. 27 is a block diagram illustrating a handwriting text search system according to an embodiment of the present disclosure;

[0087] Figure 6 FIG. 28 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0088] It is to be understood that the sizes of the various parts shown in the drawings are not necessarily to scale. Also, like reference numerals are used to denote like parts throughout the several views. DETAILED DESCRIPTION

[0089] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative in nature and is in no way intended to limit the disclosure, its application or uses. The disclosure can take many different forms and is not limited to the embodiments described herein. These embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the disclosure to those skilled in the art. It should be noted that the relative arrangement of the components and steps set forth in these embodiments should be interpreted as illustrative only, and not as limiting.

[0090] All terms used in the present disclosure, including technical or scientific terms, have the same meanings as those understood by those having ordinary knowledge in the art to which the present disclosure pertains, unless otherwise specifically defined. It should also be noted that the terms defined in commonly used dictionaries should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense, unless specifically so defined herein.

[0091] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the description if appropriate.

[0092] Figure 1 is a flowchart illustrating a handwriting text recognition method according to one embodiment of the present disclosure. As shown in Figure 1 the handwriting text recognition method includes: step S0, obtaining information sequence of handwriting text including a plurality of trajectory points; step S1, dividing the plurality of trajectory points into a plurality of strokes according to a writing state value of each trajectory point; step S2, calculating a first segmentation threshold of the handwriting text; step S3, determining a first text segmentation point according to a comparison result of a difference absolute value of horizontal coordinates of a start point trajectory point of a stroke and an end point trajectory point of a previous stroke and the first segmentation threshold; and step S4, performing text segmentation according to the first text segmentation point to obtain a text segmentation result.

[0093] The information of each trajectory point of the handwritten text obtained in step S0 can include multiple dimensions. The multiple trajectory points constitute an information sequence. For example, the information of each trajectory point can include the horizontal coordinate of the trajectory point, and can also include the vertical coordinate of the trajectory point. The information of each trajectory point can also include the writing time and the writing state value of the trajectory point. The writing state value includes a first value representing lifting of a stroke, and a second value representing landing of a stroke. For example, the writing state value of the terminal trajectory point of a stroke is the first value (for example, 1) representing lifting of the stroke, and the writing state value of the starting trajectory point of a stroke is the second value (for example, 0) representing landing of the stroke. Of course, the writing state value can also include a third value representing pressing of a stroke or running of a stroke. Thus, in step S1, the multiple trajectory points can be divided into multiple strokes according to the writing state value of each trajectory point.

[0094] In some embodiments, when writing characters on a touch screen, an information sequence including multiple trajectory points is generated. For example, when writing on an electromagnetic screen, a capacitive screen or the like, a handwritten trajectory point information sequence stored in time sequence is collected. In other embodiments, a terminal such as a conference integrated machine, a mobile phone, a smart ink tablet, obtains handwritten trajectory point sequence information generated by writing characters on a touch screen by using software, and can transmit the handwritten trajectory point sequence information to a cloud server or an embedded terminal according to actual needs of a service.

[0095] Based on the obtained trajectory point sequence, how to segment the handwritten text is described next.

[0096] A stroke represents the process from pressing of a stroke to lifting of a stroke. The segmentation point of the text cannot be in the middle of a stroke, and can only be the starting point of a stroke. A character (Chinese character, English letter or symbol) can generally be composed of multiple strokes. On the one hand, only when the distance between the starting point of a stroke and the terminal point of the previous stroke is far enough, the two strokes can be in different text line segments. On the other hand, even for two text lines written on the same page, the length of the characters is not always the same, so the distance between the starting point of a stroke and the terminal point of the previous stroke is not fixed.

[0097] Considering the uncertainty of the length of a written text line and the fact that the resolution of different touch screens is different, in order to make the segmentation of the handwritten text independent of the application scenario and independent of the hardware device, the segmentation threshold cannot be a fixed threshold, but needs to be a dynamic threshold, for example, determined according to the input handwritten text trajectory point sequence information.

[0098] In some embodiments, considering the daily writing habit and the time attribute of the handwritten trajectory point sequence, the first segmentation threshold can be calculated as a first sub-segmentation threshold according to the absolute value of the difference between the horizontal coordinates of the starting point of a stroke and the terminal point of the previous stroke.

[0099] For example, the first sub-segmentation threshold of the handwritten text is calculated according to statistical information of the difference absolute value of the horizontal coordinate of the start point trajectory point of each stroke in the handwritten text except the first stroke and the end point trajectory point of the previous stroke. The statistical information can include mean and variance.

[0100] In some embodiments, the first sub-segmentation threshold is calculated according to the sum of the mean and the variance. For example, the first sub-segmentation threshold (Thred1) = mean (X mean ) + variance (X var ).

[0101] The written text can include more characters, such as multiple lines of characters, or can include only a few characters, such as one line of characters or even one character. In view of the universality of text segmentation, for the case of very few characters, in order to improve the accuracy of text segmentation, the segmentation threshold is at least half of the overall width of the characters. In other embodiments, the first segmentation threshold includes a second sub-segmentation threshold in addition to the first sub-segmentation threshold. The second sub-segmentation threshold (Thred2) can be calculated according to the difference between the maximum value (X max ) and the minimum value (X min ) of the horizontal coordinates of the plurality of trajectory points. For example, Thred2 = (X max – X min ) × k, where k is determined according to the actual situation and takes a value of [0.5, 1]. Considering that the longest text line generally written in the actual application scenario is more than 2 characters, k can be 0.5.

[0102] In step S3, for each stroke in the handwritten text except the first stroke, it is determined whether the difference absolute value of the horizontal coordinate of the start point trajectory point of the stroke and the end point trajectory point of the previous stroke is greater than the first segmentation threshold. In the case where the difference absolute value of the horizontal coordinate of the start point trajectory point of the stroke and the end point trajectory point of the previous stroke is greater than the first segmentation threshold, the start point trajectory point is determined as the first text segmentation point. For example, each trajectory point can be traversed in turn to determine the start point and end point of each stroke. If a trajectory point is the start point of a stroke and the stroke is not the first stroke, the above determination is performed.

[0103] In some embodiments, in the case that the variance is greater than or equal to the mean, or the first sub-segmentation threshold is greater than or equal to the second sub-segmentation threshold, it is determined whether the difference absolute value of the horizontal coordinates of the start point of a stroke and the end point of the previous stroke is greater than the first segmentation threshold. Conversely, in the case that the variance is less than the mean, and the first sub-segmentation threshold is less than the second sub-segmentation threshold, it generally occurs when the sequence of the input handwritten text has only single-line data. In this case, the sequence of the input handwritten text does not need to be segmented, and thus the above determination is not needed.

[0104] The mean reflects the average change of the difference absolute value of the horizontal coordinates, and the variance reflects the degree of deviation of the difference absolute value of the horizontal coordinates from the mean. For multi-line text, the difference between the end point of the last stroke of the last character of a line and the start point of the first character of the next line is large, so the variance is large. For single-line text, the variance is small. If the single-line text is long, the first sub-segmentation threshold can be less than the second sub-segmentation threshold. The variance is limited to be less than the mean, and the first sub-segmentation threshold is limited to be less than the second sub-segmentation threshold, so that the condition for not performing segmentation is more stringent.

[0105] In step S3, for example, in the case that the difference absolute value is greater than the first sub-segmentation threshold or the second sub-segmentation threshold, the start point is determined as a first text segmentation point. In step S4, text segmentation is performed according to the determined first text segmentation point to obtain a text segmentation result. In some embodiments, each first text segmentation point can be the start point of a text segmentation unit. A text segmentation unit corresponds to a line of text. For example, for 3 lines of text, the start point of the first stroke of the second line is a text segmentation point, and the text can be segmented into the first line of text and the remaining text from the text segmentation point. Similarly, the start point of the first stroke of the third line is also a text segmentation point, and the text can be further segmented into the second line of text and the third line of text from the text segmentation point.

[0106] In some embodiments, the start point of each stroke and the end point of the previous stroke can be sequentially traversed. If the difference absolute value is greater than the first sub-segmentation threshold or the second sub-segmentation threshold, the first text segmentation point determination in step S4 is performed. Generally, for multi-line handwritten text, the calculated first sub-segmentation threshold is greater than the second sub-segmentation threshold. Therefore, here the difference absolute value can only be compared with the second sub-segmentation threshold. If the difference absolute value is less than or equal to the second sub-segmentation threshold, text segmentation is not performed.

[0107] In the above embodiments, the strokes are divided based on the writing state values of the trajectory points, and the text line segmentation is performed based on the statistical result of the absolute value difference of the horizontal coordinates of the starting point of a stroke and the ending point of the previous stroke, which can improve the accuracy of text segmentation.

[0108] In some embodiments, in order to improve the efficiency of text segmentation, the handwriting text can be first coarsely segmented based on the writing time of the trajectory points in the strokes, and then finely segmented (i.e., determining the first text segmentation points) based on the horizontal coordinates of the trajectory points and the writing state values.

[0109] For example, before determining the first text segmentation points, second text segmentation points are determined according to the comparison result of the difference between the writing times corresponding to the plurality of strokes and a second segmentation threshold (which can also be referred to as a time threshold); and then, according to the second text segmentation points, the plurality of strokes are divided into at least one text segmentation unit, and the difference between the writing times of the strokes in each text segmentation unit is less than or equal to the time threshold.

[0110] The time threshold can be determined based on the statistical information of the time difference between the strokes. Different time thresholds can be set according to different use scenarios. For example, in daily handwriting, the time threshold can be set to 10 seconds.

[0111] In other embodiments, in order to further improve the accuracy of text segmentation, especially for handwriting texts written at an angle, before determining the first text segmentation points based on the horizontal coordinates of the trajectory points, the vertical coordinates of the trajectory points can also be considered, and the handwriting text can be coarsely segmented by using the distance between the trajectory points.

[0112] For example, first, third text segmentation points are determined according to the comparison result of the distance between the starting point trajectory point of a stroke and the ending point trajectory point of the previous stroke and a third segmentation threshold (which can also be referred to as a distance threshold); and then, according to the third text segmentation points, the strokes in each text segmentation unit obtained according to the second text segmentation points are segmented to obtain a plurality of text segmentation sub-units, wherein the distance between the starting point trajectory point of a stroke in each text segmentation sub-unit and the ending point trajectory point of the previous stroke is less than or equal to the distance threshold.

[0113] It should be understood that the coarse segmentation by using the distance between the trajectory points and the coarse segmentation by using the writing time of the trajectory points can both be performed or only one of them can be performed. If both are performed, considering that the accuracy of text segmentation based on the distance between the trajectory points is higher than that of text segmentation by using the writing time of the trajectory points, the text segmentation by using the writing time of the trajectory points can be performed first, and then the text segmentation based on the distance between the trajectory points can be performed on this basis.

[0114] The distance threshold can be determined based on the average length of multiple strokes. For example, for each stroke, take three trajectory points, namely the start point, the midpoint, and the end point, calculate the distance between these three trajectory points and add them together as the length of each stroke; then calculate the average length of all strokes, and use a multiple of the average length (such as 2.5 times) as the distance threshold.

[0115] For each stroke in a text segmentation subunit, if the absolute difference between the x-coordinate of the starting trajectory point of a stroke and the ending trajectory point of the previous stroke is greater than the segmentation threshold, the starting trajectory point is determined as the first text segmentation point.

[0116] It should be understood that the calculation method for the first segmentation threshold differs in different embodiments. In this embodiment, considering the difference in stroke length between large and small characters, the first segmentation threshold is determined based on the average length of the strokes in each text segmentation subunit. For example, a multiple of the average length (such as 1.5 times) is used as the segmentation threshold here.

[0117] In some embodiments, to further improve the accuracy of text segmentation, especially to avoid splitting handwritten text on the same line into different lines (i.e., multiple text segmentation units / sub-units), when obtaining the text segmentation result based on the first text segmentation point in step S4, the various text segmentation units obtained based on the first text segmentation point can be appropriately merged. The text segmentation units can be merged based on a comparison of the overlap between the ordinate ranges of the trajectory points included in each text segmentation unit and a merging threshold to obtain the text segmentation result. For example, if the overlap between the ordinate ranges of two text segmentation units is greater than the merging threshold (e.g., 0.5), these two text segmentation units are merged into one text segmentation unit.

[0118] The preceding sections described various text segmentation methods in handwritten text recognition, including coarse segmentation, fine segmentation, and various embodiments of merging segmentation results. It should be understood that these embodiments can be combined as needed. Figure 1A One of the combinations is shown.

[0119] like Figure 1A As shown, a handwritten text segmentation method according to an embodiment of the present disclosure includes steps 120-180.

[0120] After the plurality of stroke points are divided into a plurality of strokes in step S1, the handwriting text is first coarsely segmented based on the writing time of the stroke points in the strokes in step 120. For example, according to the comparison result of the difference between the writing time of the plurality of strokes and a time threshold, the plurality of strokes are divided into at least one first stroke segmentation block. Each first stroke segmentation block corresponds to a text segmentation unit or a text line, and the difference between the writing time of the strokes in each first stroke segmentation block is less than or equal to the time threshold.

[0121] Next, the handwriting text can also be further coarsely segmented by the distance between the stroke points. That is, in step 140, according to the comparison result of the distance between the starting point of a stroke and the ending point of the previous stroke and a first distance threshold, the strokes in each first stroke segmentation block are segmented to obtain at least one second stroke segmentation block. Each second stroke segmentation block corresponds to a text segmentation sub-unit or a text line, and the distance between the starting point of a stroke in each second stroke segmentation block and the ending point of the previous stroke is less than or equal to the first distance threshold. The first distance threshold can be determined according to the average length of the plurality of strokes, for example, the first distance threshold can be 2.5 times the average length.

[0122] Then, the handwriting text is finely segmented by the horizontal distance between the stroke points (also referred to as the absolute value of the difference in the horizontal coordinate). That is, in step 160, according to the comparison result of the absolute value of the difference in the horizontal coordinate between the starting point of a stroke and the ending point of the previous stroke and a second distance threshold, the strokes in each second stroke segmentation block are segmented to obtain at least one third stroke segmentation block. Each third stroke segmentation block corresponds to a text segmentation sub-unit or a text line, and the distance between the starting point of a stroke in each third stroke segmentation block and the ending point of the previous stroke is less than or equal to the second distance threshold.

[0123] In some embodiments, the second distance threshold corresponds to the aforementioned segmentation threshold. In other embodiments, considering the difference in the length of large and small strokes, the second distance threshold is determined according to the average length of the strokes in each second stroke segmentation block, for example, the second distance threshold can be 1.5 times the average length.

[0124] Finally, to avoid splitting handwritten text from the same line into different lines, in step 180, the third-stroke segments are merged based on a comparison of the overlap between the vertical coordinate ranges of each third-stroke segment and a merging threshold, resulting in at least one fourth-stroke segment. For example, if the overlap between the vertical coordinate ranges of two third-stroke segments is greater than the merging threshold (e.g., 0.5), these two third-stroke segments are merged into one fourth-stroke segment. Each fourth-stroke segment corresponds to a text segmentation unit or a text line. The result of the fourth-stroke segment can be used as the text segmentation result of the handwritten text. Based on the above text segmentation results, handwritten text can be recognized.

[0125] In addition, the online text segmentation results described above can be combined with the offline text detection results to further improve the accuracy of text segmentation.

[0126] Figure 2 This is a flowchart illustrating a handwritten text recognition method according to another embodiment of the present disclosure. Figure 2 and Figure 1 The difference is that, Figure 2 The handwritten text recognition method also includes steps S5-S8, which further segment the text based on the offline text detection results. Only the following will describe... Figure 2 and Figure 1 The differences are minor, while the similarities will not be elaborated upon.

[0127] In step S5, a handwritten text image is formed based on the information sequence of multiple trajectory points of the handwritten text.

[0128] The following is combined with Figure 2A The process of forming a handwritten text image according to some embodiments of this disclosure is described. For example... Figure 2A As shown, the process of forming a handwritten text image includes steps S51-S54.

[0129] In step S51, the maximum value X of the x-coordinates of the multiple trajectory points is calculated. max and minimum value X min The maximum and minimum values ​​of the ordinate Y max and minimum value Y min The range of the x-coordinate X is obtained. range =X max -X min and the range of the vertical axis Y range =Y max -Y min The horizontal axis range corresponds to the width of the text, and the vertical axis range corresponds to the length of the text.

[0130] In step S52, according to the specified image size I width and I height, the horizontal coordinate range X range and the vertical coordinate range Y range , calculate the scaling factor R X = I width / X range and R Y = I height / Y range .

[0131] In step S53, the value of the horizontal coordinate and the vertical coordinate of each trajectory point is scaled according to the scaling factor R X and R Y respectively, to obtain a plurality of coordinate-scaled trajectory points.

[0132] For example, the horizontal coordinate of each trajectory point is multiplied by the scaling factor R X to obtain the scaled horizontal coordinate; the vertical coordinate of each trajectory point is multiplied by the scaling factor R Y to obtain the scaled vertical coordinate.

[0133] In step S54, the plurality of coordinate-scaled trajectory points are connected to form the handwritten text image, wherein the end point trajectory point of one stroke is not connected to the start point trajectory point of the next stroke.

[0134] Of course, the plurality of trajectory points can also be connected to form the image first, and then the image is scaled according to the specified image size and the relationship between the horizontal coordinate range and the vertical coordinate range.

[0135] Next, in step S6, an offline text detection algorithm is used to perform text detection on the handwritten text image to obtain a plurality of text detection boxes. The offline text detection algorithm can use an existing offline text detection algorithm, for example, a network trained based on ContourNet. One text detection box corresponds to one line of text, and different text detection boxes correspond to different lines of text. In other words, the text in one text detection box belongs to the same line, and the text in different text detection boxes belongs to different lines.

[0136] In step S7, the text detection box to which each trajectory point belongs is identified.

[0137] The process of identifying the text detection box to which each trajectory point belongs according to some embodiments of the present disclosure will be described below in conjunction with Figure 2B As shown in Figure 2B , the process of identifying the text detection box to which each trajectory point belongs includes steps S71-S72.

[0138] In step S71, different text regions enclosed by different text detection boxes are identified respectively, and a text detection marked image is generated. For example, a connected component filling algorithm is used for marking. The text region enclosed by one text detection box is connected, and the text regions enclosed by different text detection boxes are not connected. Different connected components can be filled with different colors. In some embodiments, the image region not belonging to any text region has a first pixel value (e.g., 0), the text region enclosed by the same text detection box has the same second pixel value (e.g., an integer greater than 0), and the text regions enclosed by different text detection boxes have different second pixel values (e.g., 1, 2, 3, …, respectively).

[0139] In step S72, according to the pixel value of the corresponding position of each trajectory point in the text detection marked image, the text detection box to which each trajectory point belongs is identified. For example, for trajectory point A, the corresponding position of trajectory point A in the text detection marked image can be determined according to the scaled horizontal coordinate and vertical coordinate of trajectory point A, and then whether trajectory point A belongs to a certain text detection box is determined according to the pixel value of the position. Specifically, in the case where the pixel value of the position is the first pixel value (e.g., 0), it is determined that trajectory point A does not belong to any text detection box; and in the case where the pixel value of the position is the second pixel value (e.g., 1), it is determined that trajectory point A belongs to the text detection box corresponding to the pixel value 1. Similarly, for trajectory point B, a similar determination can also be made. If the pixel value of the corresponding position of trajectory point B in the text detection marked image is also 1, it is determined that trajectory point B and A belong to the same text detection box, and if it is a second pixel value (e.g., 2) different from 1, it is determined that trajectory point B and A belong to different text detection boxes.

[0140] Next, text segmentation is performed according to the identified results. In step S8, according to the identified results, the text segmentation result is corrected according to the identified results.

[0141] The process of text segmentation according to the identified results according to some embodiments of the present disclosure will be described below. Figure 2C As shown in FIG. 8, the process of correcting the text segmentation result according to the identified results includes steps S81-S83. Figure 2C

[0142] In step S81, trajectory points not belonging to any text detection box are deleted. In this way, some noise can be ignored, and the adverse effects of noise on text segmentation can be avoided.

[0143] In step S82, for trajectory points belonging to text detection boxes, it is determined whether two adjacent trajectory points belong to different text detection boxes.

[0144] ​In step S83, in the case that the two adjacent trajectory points belong to different text detection boxes, the two adjacent trajectory points are segmented to obtain two text segmentation sub-units.

[0145] For example, for handwriting texts that are written first in a horizontal line and then in an inclined manner, the result of offline segmentation is more accurate. The result of offline segmentation is used to correct the result of online segmentation, so that the accuracy of text segmentation can be further improved.

[0146] In the above embodiments, the online text segmentation result is obtained according to the statistical information of the trajectory points, and the online segmentation result is corrected by using the result of offline segmentation to further improve the accuracy of text segmentation. The text recognition result is further described below according to the text segmentation result. For example, a text recognition model using BiLSTM can be used to recognize the text segmentation result to obtain the text recognition result.

[0147] The text recognition model can be composed of multiple (for example, three) BiLSTMs stacked, followed by two full connection layers for classification. The three BiLSTMs can respectively contain 256, 256 and 512 units, and the two full connection layers respectively contain 512 and 3816 neurons. Here, 3816 represents the final classification categories, including 3755 Chinese characters in the Chinese character first-level library, 26 English letters, 10 Arabic numerals and 25 commonly used punctuation marks. A Dropout layer with a probability of 0.5 is also used after each LSTM to randomly delete part of the connections to reduce model overfitting and speed up model convergence.

[0148] In some embodiments, in order to improve the accuracy of text recognition, the inclined text line (i.e., text segmentation unit) can be rotated and transformed to the common left-to-right writing direction (i.e., horizontal direction) before the text segmentation result is input into the text recognition model. The description is combined with the schematic diagram of the inclined text line as shown in Figure 2D

[0149] As Figure 2D As shown, the angle θ of each text segmentation unit relative to the horizontal direction can be obtained by using the minimum bounding rectangle algorithm; and the trajectory points included in each text segmentation unit are rotated and transformed according to the angle θ of each text segmentation unit. Specifically, the connected domain in the text line can be calculated according to the trajectory points in the text line, for example, the trajectory points in the text segmentation unit are connected and then the angle of the text line relative to the horizontal direction is obtained according to the minimum bounding rectangle algorithm, so as to rotate and transform the text line to the horizontal direction. The minimum bounding rectangle can be defined by coordinates (X1, Y1), (X2, Y2), (X3, Y3) and (X4, Y4).

[0150] ​In some embodiments, in order to improve the efficiency of text recognition, redundant trajectory points can be deleted before text recognition to reduce the amount of data to be recognized, thereby improving the recognition speed. For example, the slope of the line connecting each trajectory point in a stroke with its previous and next trajectory points is calculated; for each trajectory point in a stroke, if the difference between the slopes of the lines connecting the trajectory point with its previous and next trajectory points is less than or equal to a change threshold, then the trajectory point is deleted. If the difference between the slopes of the lines connecting a trajectory point with its previous and next trajectory points is less than or equal to a change threshold (e.g., 0.1), then the three trajectory points can be considered to be on a straight line, and thus the trajectory point can be deleted to reduce the amount of data to be recognized.

[0151] In yet some embodiments, in order to improve the effect of text recognition, considering that the coordinate range of trajectory points in each text line can be quite different, the trajectory points of each text line can be normalized before text recognition. For example, the vertical coordinates of the trajectory points included in each text segmentation unit can be normalized, e.g., to the range of 0-1, while the horizontal coordinates are scaled in proportion to the vertical coordinates.

[0152] The embodiments of the present disclosure also provide a method for searching handwritten text, comprising: obtaining an information sequence of the handwritten text to be searched, the information sequence comprising a plurality of trajectory points; dividing the plurality of trajectory points into a plurality of strokes; recognizing the handwritten text to be searched by using a stroke-based handwritten text recognition model to obtain a text recognition result; and performing corresponding search in the obtained text recognition result in response to a search command input by a user.

[0153] In some embodiments, recognizing the handwritten text to be searched to obtain a text recognition result comprises: recognizing the handwritten text to be searched according to the handwritten text recognition method of any of the preceding embodiments to obtain a text recognition result.

[0154] So far, various method embodiments of the present disclosure have been described in detail, and corresponding product embodiments are described below. The embodiments of the present disclosure also provide a handwritten text recognition device.

[0155] Figure 3A is a block diagram illustrating a handwritten text recognition device according to an embodiment of the present disclosure.

[0156] As Figure 3A shown, the handwritten text recognition device 30 comprises an obtaining unit 300, a dividing unit 310, a calculating unit 320, a determining unit 330, and a segmentation unit 340.

[0157] The obtaining unit 300 is configured to obtain an information sequence of the handwritten text, the information sequence comprising a plurality of trajectory points, the information of each trajectory point comprising the horizontal coordinate, writing time, and writing state value of the trajectory point, e.g., performing step S0.

[0158] The dividing unit 310 is configured to divide the plurality of trajectory points into a plurality of strokes according to the writing state value of each trajectory point, for example, performing step S1.

[0159] The calculating unit 320 is configured to calculate a first segmentation threshold of the handwritten text, for example, performing step S2.

[0160] The determining unit 330 is configured to determine a first text segmentation point according to a comparison result of a difference absolute value of horizontal coordinates of a start point trajectory point of a stroke and an end point trajectory point of a previous stroke and the first segmentation threshold, for example, performing step S3.

[0161] The segmenting unit 340 is configured to perform text segmentation according to the first text segmentation point to obtain a text segmentation result, for example, performing step S4.

[0162] Figure 3B is a block diagram illustrating a handwritten text recognition apparatus according to another embodiment of the present disclosure. Figure 3B Different from Figure 3A , the handwritten text recognition apparatus 30' in Figure 3B further comprises an image forming unit 350, a text detecting unit 360, an identifying unit 370 and a text segmenting unit 380. Only Figure 3B will be described below, and the same parts will not be described again. Figure 3A Different from

[0163] The image forming unit 350 is configured to form a handwritten text image according to information sequence of the handwritten text comprising a plurality of trajectory points, for example, performing step S5.

[0164] The text detecting unit 360 is configured to perform text detection on the handwritten text image by using an offline text detection algorithm to obtain a plurality of text detection boxes, for example, performing step S6.

[0165] The identifying unit 370 is configured to identify a text detection box to which each trajectory point belongs, for example, performing step S7.

[0166] The text segmenting unit 380 is configured to correct the text segmentation result according to the identification result, for example, performing step S8.

[0167] Figure 4 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0168] As Figure 4As shown, the electronic device 40 comprises a memory 410 and a processor 420 coupled to the memory 410. The memory 410 is configured to store instructions for implementing an embodiment of the method of handwritten text recognition. The processor 420 is configured to execute one or more steps of the method of handwritten text recognition or the method of handwritten text search in any of the embodiments of the present disclosure based on the instructions stored in the memory 410. That is, the electronic device 40 can be a handwritten text recognition apparatus or a handwritten text search apparatus.

[0169] It should be appreciated that one or more steps of the method of handwritten text recognition can be implemented by a processor and can be implemented in any of software, hardware, firmware or a combination thereof.

[0170] In addition to the method and apparatus of handwritten text recognition, the method and apparatus of handwritten text search, the embodiments of the present disclosure can also take the form of a computer program product implemented on one or more non-transitory storage media containing computer program instructions. Therefore, the embodiments of the present disclosure also provide a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement one or more steps of the method of handwritten text recognition or the method of handwritten text search in any of the aforementioned embodiments.

[0171] Figure 5A is a block diagram showing a handwritten text recognition system according to an embodiment of the present disclosure.

[0172] As shown in Figure 5A , the handwritten text recognition system 5A comprises a touch screen 50 configured to generate a sequence of information comprising a plurality of trajectory points when a user writes characters on the touch screen, and the aforementioned handwritten text recognition apparatus 30, 30' or 40.

[0173] In some embodiments, the handwritten text recognition system 5A further comprises a terminal 60 configured to obtain the sequence of information comprising a plurality of trajectory points generated when a user writes characters on the touch screen and send it to the handwritten text recognition apparatus.

[0174] As mentioned before, the handwritten text recognition apparatus 30, 30' or 40 can be located at a cloud server or an embedded terminal, perform handwritten text segmentation on the received sequence of handwritten trajectory point information, and transmit the text segmentation result to a designated recipient, such as a cloud or a terminal. The text segmentation result can be used for subsequent handwritten text recognition.

[0175] Figure 5B is a block diagram showing a handwritten text search system according to an embodiment of the present disclosure.

[0176] As shown in Figure 5BAs shown, the handwritten text search system 5 includes the aforementioned handwritten text recognition system 5A configured to recognize the handwritten text to be searched and obtain a text recognition result, and a search engine 5B configured to perform a corresponding search in the obtained text recognition result in response to a search command input by a user.

[0177] Figure 6 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0178] As Figure 6 As shown, the computer system can be in the form of a general-purpose computing device, which can be used to implement the handwritten text recognition apparatus of the above-described embodiments. The computer system includes a memory 610, a processor 620, and a bus 600 connecting different system components.

[0179] The memory 610 can include, for example, a system memory, a non-volatile storage medium, and the like. The system memory, for example, stores an operating system, application programs, a Boot Loader, and other programs, and the like. The system memory can include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium, for example, stores instructions for implementing corresponding embodiments of the display method. The non-volatile storage medium includes, but is not limited to, a magnetic disk storage, an optical disk storage, a flash memory, and the like.

[0180] The processor 620 can be implemented in the form of a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete hardware component such as a discrete gate or transistor, and the like. Accordingly, each device such as a determination device and a determination device can be implemented by a central processing unit (CPU) running instructions in the memory for performing corresponding steps, or by a dedicated circuit for performing corresponding steps.

[0181] The bus 600 can use any of a variety of bus structures. For example, the bus structure includes, but is not limited to, an industry standard architecture (ISA) bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus.

[0182] The computer system can also include an input / output interface 630, a network interface 640, a storage interface 650, and the like. These interfaces 630, 640, 650 and the memory 610 and the processor 620 can be connected through the bus 600. The input / output interface 630 can provide a connection interface for display, mouse, keyboard, and the like input / output devices. The network interface 640 provides a connection interface for various networking devices. The storage interface 640 provides a connection interface for external storage devices such as floppy disks, U disks, SD cards, and the like.

[0183] So far, various embodiments of the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein according to the above description.

[0184] Although some specific embodiments of the present disclosure have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration, not for limiting the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified or some technical features can be replaced equivalently without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A handwriting text recognition method, comprising: obtaining an information sequence of a handwriting text comprising a plurality of trajectory points, information of each trajectory point comprising a horizontal coordinate, a writing time and a writing state value of the trajectory point; dividing the plurality of trajectory points into a plurality of strokes according to the writing state value of each trajectory point, the writing state value comprising a first value representing lifting of a stroke and a second value representing landing of the stroke respectively; calculating a first segmentation threshold of the handwriting text; determining a first text segmentation point according to a comparison result of a difference absolute value of horizontal coordinates of a start point trajectory point of a stroke and an end point trajectory point of a previous stroke with the first segmentation threshold; performing text segmentation according to the first text segmentation point to obtain a text segmentation result.

2. The handwriting text recognition method of claim 1, wherein, determining the first text segmentation point comprises: in a case that the difference absolute value of horizontal coordinates of the start point trajectory point of the stroke and the end point trajectory point of the previous stroke is greater than the first segmentation threshold, determining the start point trajectory point as the first text segmentation point.

3. The handwriting text recognition method according to claim 1 or 2, wherein, the first segmentation threshold comprises a first sub-segmentation threshold, and calculating the first segmentation threshold of the handwriting text comprises: calculating the first sub-segmentation threshold of the handwriting text according to statistical information of the difference absolute value of horizontal coordinates of a start point trajectory point of each stroke except a first stroke and an end point trajectory point of a previous stroke.

4. The handwriting recognition method of claim 3, wherein, the statistical information comprises a mean value and a variance, and calculating the first sub-segmentation threshold of the handwriting text comprises: calculating the first sub-segmentation threshold according to a sum of the mean value and the variance.

5. The handwriting recognition method of claim 3, wherein, the first segmentation threshold further comprises a second sub-segmentation threshold, and calculating the first segmentation threshold of the handwriting text comprises: calculating the second sub-segmentation threshold according to a difference between a maximum value and a minimum value of the horizontal coordinates of the plurality of trajectory points.

6. The handwriting recognition method of claim 5, wherein, the statistical information comprises a mean value and a variance, and determining the first text segmentation point comprises: in a case that the variance is greater than or equal to the mean value, or the first sub-segmentation threshold is greater than or equal to the second sub-segmentation threshold, judging whether the difference absolute value of horizontal coordinates of a start point trajectory point of a stroke and an end point trajectory point of a previous stroke is greater than the first segmentation threshold.

7. The handwriting recognition method of claim 5, wherein, in a case that the difference absolute value is greater than the first segmentation threshold, determining the start point trajectory point as the first text segmentation point comprises: in a case that the difference absolute value is greater than the first sub-segmentation threshold or the second sub-segmentation threshold, determining the start point trajectory point as the first text segmentation point.

8. The method of handwriting text recognition according to any one of claims 1 to 7, wherein, the handwriting text recognition method further comprises: before determining the first text segmentation point, determining a second text segmentation point according to a comparison result of a difference of writing times corresponding to the plurality of strokes with a second segmentation threshold; dividing the plurality of strokes into at least one text segmentation unit according to the second text segmentation point, a difference of writing times between strokes in each text segmentation unit being less than or equal to the second segmentation threshold.

9. The handwriting recognition method of claim 8, wherein, information of each trajectory point further comprises a vertical coordinate of the trajectory point, and the handwriting text recognition method further comprises: before determining the first text segmentation point, determining a third text segmentation point according to a comparison result of a distance of a start point trajectory point of a stroke and an end point trajectory point of a previous stroke with a third segmentation threshold; According to the third text segmentation point, strokes in each text segmentation unit obtained according to the second text segmentation point are segmented to obtain a plurality of text segmentation sub-units, wherein a start point track point of a stroke in each text segmentation sub-unit and an end point track point of a previous stroke have a distance less than or equal to the third segmentation threshold.

10. The handwriting recognition method of claim 9, wherein, The third segmentation threshold is determined according to average lengths of the plurality of strokes.

11. The handwriting recognition method of claim 9, wherein, The first segmentation threshold is determined according to average lengths of strokes in each text segmentation sub-unit.

12. The method of handwriting text recognition according to any one of claims 1 to 11, wherein, The text segmentation according to the first text segmentation point comprises: Merging text segmentation units according to a comparison result of an overlapping degree between vertical coordinate ranges of track points included in each text segmentation unit obtained according to the first text segmentation point and a merging threshold to obtain the text segmentation result.

13. The handwriting text recognition method according to any one of claims 1 to 12, further comprising: forming a handwriting text image according to information sequences of the handwriting text including a plurality of track points; performing text detection on the handwriting text image by using an offline text detection algorithm to obtain at least one text detection frame; identifying a text detection frame to which each track point belongs; correcting the text segmentation result according to the identification result.

14. The handwriting recognition method of claim 13, wherein, The correcting the text segmentation result according to the identification result comprises: deleting track points not belonging to any text detection frame; for track points belonging to a text detection frame, judging whether two adjacent track points belong to different text detection frames; in a case where two adjacent track points belong to different text detection frames, segmenting the two adjacent track points to obtain two text segmentation sub-units.

15. The handwriting recognition method according to claim 13 or 14, wherein, The identifying a text detection frame to which each track point belongs comprises: identifying different text regions surrounded by different text detection frames respectively to generate a text detection identification image, wherein image regions not belonging to any text region in the text detection identification image have a first pixel value, text regions surrounded by a same text detection frame have a same second pixel value, and text regions surrounded by different text detection frames have different second pixel values; identifying a text detection frame to which each track point belongs according to a pixel value of a corresponding position of the track point in the text detection identification image.

16. The handwriting recognition method of claim 13 or 14, wherein, The information of each track point further comprises a vertical coordinate of the track point, and the forming a handwriting text image according to information sequences of the handwriting text including a plurality of track points comprises: statistically obtaining a maximum value Xmax and a minimum value Xmin of horizontal coordinates of the plurality of track points, and a maximum value Ymax and a minimum value Ymin of vertical coordinates of the plurality of track points to obtain a horizontal coordinate range Xrange = Xmax - Xmin and a vertical coordinate range Yrange = Ymax - Ymin; calculating scaling factors RX = Iwidth / Xrange and RY = Iheight / Yrange according to specified image sizes Iwidth and Iheight, the horizontal coordinate range Xrange and the vertical coordinate range Yrange; and scaling the information sequences of the handwriting text according to the scaling factors RX and RY to obtain the handwriting text image. scaling the horizontal coordinate and the vertical coordinate of each trajectory point according to the scaling coefficients RX and RY respectively, to obtain a plurality of trajectory points after coordinate scaling; connecting the plurality of trajectory points after coordinate scaling to form the handwritten text image, wherein an end point trajectory point of one stroke is not connected with a start point trajectory point of a next stroke.

17. The handwritten text recognition method of any one of claims 1-16, further comprising: deleting a trajectory point in each stroke if a difference between a slope of a line connecting the trajectory point and a previous trajectory point of the trajectory point and a slope of a line connecting the trajectory point and a next trajectory point of the trajectory point is less than or equal to a change threshold.

18. The handwritten text recognition method of any one of claims 1-17, further comprising: normalizing the vertical coordinate of the trajectory points included in each text segmentation unit, and scaling the horizontal coordinate proportionally to the vertical coordinate.

19. The handwritten text recognition method of any one of claims 1-18, further comprising: obtaining an angle of each text segmentation unit relative to a horizontal direction using a minimum bounding rectangle algorithm; performing a rotation transformation on the trajectory points included in each text segmentation unit according to the angle of the text segmentation unit.

20. The handwritten text recognition method of any one of claims 1-19, further comprising: recognizing the text segmentation result using a text recognition model based on bidirectional long short-term memory (BiLSTM) to obtain a text recognition result.

21. A handwritten text search method, comprising: obtaining an information sequence of a handwritten text to be searched, the information sequence including a plurality of trajectory points; dividing the plurality of trajectory points into a plurality of strokes; recognizing the handwritten text to be searched using a stroke-based handwritten text recognition model according to the handwritten text recognition method of any one of claims 1-20 to obtain a text recognition result; performing a corresponding search in the obtained text recognition result in response to a search command input by a user.

22. A handwritten text recognition apparatus, comprising: an obtaining unit configured to obtain an information sequence of a handwritten text, the information sequence including a plurality of trajectory points, information of each trajectory point including a horizontal coordinate, a writing time, and a writing state value of the trajectory point; a dividing unit configured to divide the plurality of trajectory points into a plurality of strokes according to the writing state value of each trajectory point, the writing state value including a first value representing a pen up of a stroke and a second value representing a pen down of the stroke; a calculating unit configured to calculate a first segmentation threshold of the handwritten text; a determining unit configured to determine a first text segmentation point according to a comparison result of a difference absolute value of the horizontal coordinate of a start point trajectory point of a stroke and a horizontal coordinate of an end point trajectory point of a previous stroke and the first segmentation threshold; and a segmentation unit configured to perform text segmentation according to the first text segmentation point to obtain a text segmentation result.

23. A handwritten text recognition system, comprising: a touch screen configured to generate an information sequence including a plurality of trajectory points when a user writes characters on the touch screen; a handwritten text recognition apparatus configured to perform the handwritten text recognition method of any one of claims 1-20.

24. The handwritten text recognition system of claim 23, further comprising: a terminal configured to acquire a sequence of information including a plurality of trajectory points generated when writing a character on the touch screen and send to the handwriting text recognition device.

25. A handwriting text search system, comprising: the handwriting text recognition system according to claim 23 or 24, configured to recognize handwriting text to be searched and obtain a text recognition result; a search engine configured to perform a corresponding search in the obtained text recognition result in response to a search command input by a user.

26. An electronic device, comprising: a memory; and a processor coupled to the memory, the processor configured to perform one or more steps of the handwriting text recognition method according to any one of claims 1 to 20, or one or more steps of the handwriting text search method according to claim 21, based on instructions stored in the memory device.

27. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements one or more steps of the handwriting text recognition method according to any one of claims 1 to 20, or one or more steps of the handwriting text search method according to claim 21.

Images