Information system based on automatic line breaks in handwritten documents

Abstract

To provide an information system based on automatic line feed in a handwritten document.SOLUTION: When a user actually writes, a system calculates a standard height and a standard width of a handwritten original. The system detects and deletes an abnormal stroke to prevent a situation where the abnormal stroke affects a division and an identification result obtained thereafter. The system then roughly divides the handwritten original into a plurality of lines using a pre-division unit. The system sets an isolated point and an indirect stroke as one line, then re-divides a line that is too large, and performs line alignment and spatial merging on a result obtained when the division is completed. When a word interval in the middle of the line is too large, the system divides the line into a plurality of lines again and improves accuracy of the division. Finally, the system determines, on the basis of whether the whole is a word, whether to add a second division scheme. The present invention takes into account various situations that occur when an ordinary user writes by hand, such as writing style issues of the character size, the isolated point, and the indirect stroke, and a program of the present invention appropriately handles these writing style issues, thereby improving efficiency of line feed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to line break processing of electronic handwritten documents, and more particularly to an information system based on automatic line break processing of handwritten documents. [Background technology]

[0002] In the past, when a user wrote characters on an electronic writing panel, the characters written by the user were converted into computer text by a program on the computer side, and the program arranged them in order up to the boundary where they could be written, and then automatically performed a line break.

[0003] Handwritten character segmentation technology is a technology that primarily separates letters and numbers in handwritten text. It allows computers to identify and interpret handwritten text, supporting the application of machine learning and natural language processing. Its main principle is to use various data processing techniques to analyze the characteristics of handwritten characters, such as their shape, size, spelling, and continuity, and then identify each character based on these characteristics.

[0004] Regarding handwritten character segmentation, the prior art mainly includes deep learning methods, traditional computer vision methods, and unsupervised learning methods. Based on deep learning methods, a customized convolutional neural network (CNN) with end-to-end capabilities is used to achieve the character segmentation task. Summary of the Invention [Problem to be solved by the invention]

[0005] However, traditional computer vision methods typically use a full image processing pipeline to extract multiple features from handwritten text, and then use these features to classify and identify text.Unsupervised learning methods are methods that accomplish the character segmentation task with only a single image, without requiring any tag information or a set of pre-labeled images for training.

[0006] The present invention mainly solves the problem that when processing handwritten input on an electronic writing panel, the line break of handwritten input characters is rationalized, mainly due to people's handwriting input habits and external factors when writing a large amount of text, the actual stroke order is not the same as the stroke order of the characters in the handwritten manuscript, or the strokes themselves are irregular, making it difficult to break the lines.

[0007] Therefore, the present inventors believed that the above drawbacks could be improved, and as a result of extensive research, they came up with the proposal of the present invention, which effectively improves the above problems through rational design.

[0008] The present invention has been made in view of the above circumstances, and its object is to provide an information system based on automatic line feeds for handwritten documents. [Means for solving the problem]

[0009] In order to solve the above problems, the present invention employs the following means. An information system based on automatic line breaks for handwritten documents according to one aspect of the present invention performs line breaks on a handwritten manuscript generated by a writer writing on an electronic writing panel, the system records each stroke (i.e., each coordinate point of the stroke) and the time when each coordinate point was generated, and hereinafter, the direction of the line when writing is referred to as the row direction (x direction), and the direction of skipping lines is referred to as the column direction (y direction), and the system: A processor and a memory, the processor being used to perform operations required for the system, and the memory being used to store data or operation programs of the computer program of the system, related operation results and related data; a line height unit used to calculate the standard height and standard width of the handwritten document; an abnormality deletion unit connected to the line height unit and receiving the standard height and standard width calculated by the line height unit, the abnormality deletion unit preliminarily deleting abnormal strokes from the handwritten document, and determining an abnormal stroke as an abnormal stroke when the height of the stroke is higher than the standard height by a predetermined multiple used for abnormal deletion, or when the width of the stroke is higher than the predetermined multiple used for abnormal deletion of the standard width; a pre-division unit connected to the anomaly deletion unit and receiving the processed handwritten document from the anomaly deletion unit, the pre-division unit first divides the handwritten document into a plurality of lines, the pre-division method is according to the line direction of writing, and for the first contacted line, it detects its leftmost end and rightmost end, that is, from left to right, the first contact point of the line is the leftmost end, and the last contact point of the line is the rightmost end, and then, the pre-division unit takes the set multiple used for pre-division of the standard height as the line height and pre-divides it to obtain one line, and for the remaining handwritten document, it obtains each line in order according to the same method; a special line unit connected to the pre-division unit, which first sets each isolated point and each incomplete stroke of the handwritten document after the pre-division to one line by itself, and these preset lines are merged into other lines in the following steps; an oversized line dividing unit connected to the special line unit, the oversized line dividing unit being used to re-divide the oversized line divided by the previous dividing unit, and the definition of the oversized line is that if the height of the line divided by the previous dividing unit is higher than a set multiple used for the oversized line of the standard height, the oversized line dividing unit is determined to be an oversized line; a space merging unit connected to the oversized line dividing unit, for receiving the handwritten manuscript processed by the oversized line dividing unit, for determining whether a new line is established after dividing the oversized line, and if not, for abandoning the division and accepting the original oversized line as one line, and for merging the lines formed by the isolated points and the overwritten strokes into other lines that are close to each other; a multi-line unit connected to the space merging unit and receiving the handwritten manuscript processed by the space merging unit, the multi-line unit calculating the distance between strokes in each of the lines that have already been divided, the distance being the difference between the rightmost stroke of the previous stroke and the leftmost stroke of the next stroke, and if the distance is greater than a set width for the lines, the multi-line unit treating the two strokes as a boundary and integrating the previous and next strokes into two different lines, A line break is performed on the handwritten manuscript input based on the above method. [Effects of the Invention]

[0010] As described above, the present invention has the following advantages. The present invention takes into account various situations that occur when ordinary users write by hand, such as character size, isolated points, and incomplete strokes, and the program of the present invention properly handles these writing issues to improve line break efficiency. The present invention also takes into account problems that occur during the initial line break, such as splitting multiple lines into one, and corrects these issues to improve the accuracy of line breaks. The present invention realizes line splitting of handwritten multi-line characters and achieves favorable test results, which was not possible with prior art.

[0011] Other objects, configurations and effects of the present invention will become apparent from the following detailed description of the preferred embodiments of the present invention. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating an information system based on automatic line breaks in handwritten documents according to an embodiment of the present invention. [Figure 2] 1 is a process flowchart showing an information system based on automatic line breaks in handwritten documents according to an embodiment of the present invention. [Figure 3A] FIG. 10 is a diagram illustrating the distance between each point in a stroke and the Y axis according to an embodiment of the present invention. [Figure 3B] FIG. 10 is a distribution diagram of the relationship between all stroke points and Dy in a handwritten document according to an embodiment of the present invention. [Figure 4] 1 is a schematic diagram showing x and y directions of a handwritten document according to an embodiment of the present invention; [Figure 5] 1 is a schematic diagram illustrating an information system based on automatic line breaks in handwritten documents according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0013] The following describes in detail the embodiments of the present invention, but the present invention is not limited to these, and various modifications are possible within the scope of the description, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0014] First, an information system 1 based on automatic line feeds for handwritten documents according to the present invention will be described in detail with reference to the accompanying drawings 1 to 5 as appropriate.

[0015] In this specification, a handwritten manuscript is a digital handwritten manuscript, i.e., a handwritten manuscript generated by a writer writing on the electronic writing panel 8 (see step 110). At this time, the electronic writing panel 8 records each stroke (i.e., the coordinate points of each stroke) and the time when each coordinate point was generated, so that the time sequence in which each stroke and its coordinate points were input can be known. Hereinafter, in this specification, the row direction of the handwritten manuscript is arranged on the x-axis of the coordinate system, and the column direction is arranged on the y-axis of the coordinate system (see FIG. 4).

[0016] The information system 1 of the present invention that solves the above problems mainly comprises the following components, each of which will be described below.

[0017] <Host 100> The host 100 includes a processor 2 and a memory 6. The processor 2 is used to execute the operations required by the system 1 (see FIG. 1). The memory 6 is used to store the computer program data or operation program of the system 1, related operation results, and related data. All operation results and related data are stored in the memory 6.

[0018] <Line height unit 10> The strokes of all the handwritten manuscripts are arranged based on the height of the strokes, and then the height of the strokes of a certain proportional value is set as the standard height. Similarly, the strokes of all the handwritten manuscripts are arranged based on the width of the strokes, and then the width of the strokes of a certain proportional value is set as the standard width. For example, if a handwritten manuscript has a total of 100 strokes, then the strokes are arranged from shortest to tallest based on the height of the strokes, and the 89th tallest stroke is set as the standard height. Similarly, the 100 strokes of the handwritten manuscript are arranged from shortest to longest based on the width of the strokes, and the 89th widest stroke is set as the standard width (see step 115).

[0019] <Anomalous Deletion Unit 20> The anomaly removal unit 20 is connected to the line height unit 10 and receives the standard height and standard width calculated by the line height unit 10. The anomaly removal unit 20 pre-removes abnormal strokes from handwritten documents (see step 120). The definition of an abnormal stroke is that a stroke whose height is higher than a certain multiple of the standard height or whose width is higher than a certain multiple of the standard width is regarded as an abnormal stroke. For example, if the height of a stroke is higher than five times the standard height, this stroke is regarded as an abnormal stroke and pre-removed to prevent the abnormal stroke from affecting subsequent system processing.

[0020] <Front split unit 30> The pre-division unit 30 is connected to the anomaly removal unit 20 and receives the handwritten document after processing by the anomaly removal unit 20. First, the pre-division unit 30 roughly pre-divides the handwritten document into multiple lines along the y-axis. It detects the leftmost and rightmost ends of the first touched line, i.e., from left to right, the first touched point of the line is the leftmost end, and the last touched point of the line is the rightmost end. Then, it pre-divides the first line by setting the line height to a predetermined multiple N1 of the standard height. For the remaining handwritten document, it uses the same method to obtain each successive line (see step 125).

[0021] The pre-division conditions will now be described. Condition 1: Subtract the maximum x-axis value of the conventional strokes that are larger than the standard height from the maximum x-axis value of all strokes in the current line. Condition 2: Subtract the maximum y-axis value of all strokes in the current line that are greater than twice the standard height from the minimum y-axis value of the previous stroke. Condition 3: The minimum value of the y-axis of the conventional stroke is greater than the maximum value of the y-axis of the deviation offset. The definition of the deviation offset (offset) is as follows: the minimum value of the x-axis of the conventional stroke is set as xmin, and the maximum value is set as xmax. The maximum value of the y-axis of all strokes within the (xmin, xmax) range of the current line must be less than the offset formed by the line with the minimum value of the y-axis of the conventional stroke.

[0022] If either condition 1 and condition 3 or condition 2 and condition 3 is satisfied, the current stroke is determined to belong to the next line.

[0023] The difference between the rightmost side of the previous line and the leftmost side of the next line in two consecutive lines is calculated as the horizontal axis difference dx, and the difference between the bottom of the first character of the previous line and the top of the first character of the next line in two consecutive lines is calculated as the height difference dy. The horizontal axis difference dx must be greater than the set multiple of the standard width (e.g., 5 times). The height difference dy must be greater than 0. In this case, the two lines are separated. The clearer the shift amount of the x-axis between the two lines, the more accurate the separation. If dx and dy do not satisfy the above conditions, the two consecutive lines are recognized as a single line.

[0024] <Special Unit 40> The unit is connected to the pre-segmentation unit 30 and receives the handwritten document segmented by the pre-segmentation unit 30. First, the isolated points and incomplete strokes are set to be in a single line, and these lines are merged with other lines in the following steps (see step 130).

[0025] The definition of the isolated point is as follows: The y-axis distance Dy is obtained by subtracting the y-axis value of the midpoint of a stroke in the y-axis direction from the y-axis value of the point in the stroke. This method is used to calculate the y-axis distance for all points of all strokes in the handwritten manuscript, and these y-axis distances Dy are distributed to obtain a distribution close to the normal distribution (see Figures 3A and 3B). If all points of a stroke are outside 95.449974% of this distribution (i.e., outside two standard deviations), the stroke is considered an isolated point and the stroke is set to a line by itself.

[0026] The input handwritten manuscript also records the time sequence in which each stroke was written. Therefore, the inverted stroke is defined as a difference in the y-axis direction of the line to which the stroke and the currently written stroke belong when writing that is greater than N5 times the standard height. This difference is set as the distance between the lowest point on the y-axis of the stroke and the highest point on the y-axis of the line to which the currently written stroke belongs.

[0027] <Line division unit 50 too large> The oversized line dividing unit 50 is connected to the extra line unit 40 and the pre-dividing unit 30, and receives the handwritten document after being processed by the extra line unit 40. The oversized line dividing unit 50 is used to re-divide the oversized lines divided by the pre-dividing unit 30 (see step 135).

[0028] The definition of an oversized line is that if the height (y-axis direction) of a line after division by the pre-division unit 30 is higher than the set multiple N2 of the standard height, the line is deemed to be oversized. In this case, a clustering algorithm (e.g., k-means algorithm) is applied to perform a clustering calculation along the coordinate axis (y-axis) of the height of each stroke, classifying each stroke based on its height, and then breaking each stroke (character) into a new line based on the calculation result. Thus, the oversized line is re-divided into multiple lines. After determining the oversized line, the divided lines divided from the oversized line and the original divided lines are rearranged, so that the rearranged line order matches the arrangement order of the original handwritten document (see step 140).

[0029] <Spatial Merging Unit 60> The space merging unit 60 is connected to the oversized line dividing unit 50 and receives the handwritten document processed by the oversized line dividing unit 50. The space merging unit 60 calculates the horizontal axis difference dx and the height difference dy for the new line after dividing the oversized line, and determines whether such a line break is valid. If not, it abandons the division and receives the original oversized line as a single line (see step 145).

[0030] Furthermore, the lines formed by the isolated points and the overdrawn strokes are merged into other lines based on the above calculation.

[0031] <Multi-line unit 70> The multi-line unit 70 is connected to the spatial merging unit 60 and receives the handwritten document processed by the spatial merging unit 60. The multi-line unit 70 calculates the distance between strokes in each divided line, where the distance is the difference between the rightmost stroke of the previous stroke and the leftmost stroke of the next stroke. If the distance is greater than the set width N7, the two strokes are considered to be a boundary, and the previous and next strokes are merged into two different lines (see step 150).

[0032] <All Words Judgment Unit 80> The line height unit 10 is connected to the line height unit 10, receives the standard height and standard width calculated by the line height unit 10, and determines whether the handwritten document has multiple lines (see step 155). If it does, it receives the line break result described above (see step 160). If it does not have multiple lines, it determines whether it is a single character (see step 165). The height and width of the handwritten document are calculated, and if the height is less than N3 times (e.g., 3 times) the standard height described above, or if the width is less than N4 times (e.g., 3 times) the standard width described above, it determines that the handwritten document is a single character. The aspect ratio is used as the basis for this determination. For example, if the aspect ratio is less than 2, it determines that the handwritten document is a single character (see step 170). If it is not a word, it determines that the entire text is one line (see step 175).

[0033] <Test Unit> It is used to test all other manuscripts.

[0034] As the user writes, the system calculates the standard height and width of the handwritten document. Abnormal strokes are detected and removed to prevent them from affecting the segmentation and subsequent recognition results. Next, a pre-segmentation unit is used to roughly segment the handwritten document into multiple lines. The isolated points and incomplete strokes are set to a single line. Lines that are too large are then re-segmented, and line alignment and spatial merging are performed on the segmentation results again. If the spacing between words in the middle of a line is too large, the line is again segmented into multiple lines, improving the accuracy of the line segmentation. Finally, a second segmentation method is determined based on whether the entire text is a word.

[0035] The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the present invention includes configurations that are substantially identical to the configurations described in the embodiments. A substantially identical configuration means, for example, a configuration with the same function, method, and result, or a configuration with the same purpose and effect. The present invention also includes configurations in which non-essential parts of the configurations described in the embodiments are replaced. The present invention also includes configurations that achieve the same effects or purposes as the configurations described in the embodiments. The present invention also includes configurations in which publicly known technology is added to the configurations described in the embodiments. [Explanation of symbols]

[0036] 1 System 2 processors 6. Memory 8 Electronic writing panel 10 line height units 20 Anomalous Deleted Units 30 Front split unit 40 Special Line Unit 50 too large line dividing units 60 Space Merging Unit 70 Multi-line units 80 All-word decision unit 100 hosts

Claims

1. An information system based on automatic line breaks for handwritten documents that performs line breaks on handwritten manuscripts generated by a writer writing on an electronic writing panel, the information system records each stroke (i.e., each coordinate point of the stroke) and the time when each coordinate point was generated, and hereinafter, the direction of the line when writing is referred to as the row direction (x direction), and the direction of skipping lines is referred to as the column direction (y direction), and the information system: A processor and a memory, the processor being used to perform operations required for the information system, and the memory being used to store data or operation programs of the computer programs of the information system, related operation results and related data; a line height unit used to calculate the standard height and standard width of the handwritten document; an abnormality deletion unit connected to the line height unit and receiving the standard height and standard width calculated by the line height unit, the abnormality deletion unit preliminarily deleting abnormal strokes from the handwritten document, and determining an abnormal stroke as an abnormal stroke when the height of the stroke is higher than the standard height by a predetermined multiple used for abnormal deletion, or when the width of the stroke is higher than the predetermined multiple used for abnormal deletion of the standard width; a pre-division unit connected to the anomaly deletion unit and receiving the processed handwritten document from the anomaly deletion unit, the pre-division unit first divides the handwritten document into a plurality of lines, performs pre-division according to the line direction to be written, detects the leftmost end and the rightmost end of the first contacted line, that is, from left to right, the first contact point of the line is the leftmost end, and the last contact point of the line is the rightmost end, then takes the set multiple used for pre-division of the standard height as the line height and pre-divides it to obtain one line, and obtains each successive line of the remaining handwritten document in the same manner; a special line unit connected to the pre-division unit, which first sets each isolated point and each incomplete stroke of the handwritten document after the pre-division to one line by itself, and these preset lines are merged into other lines in the following steps; an oversized line dividing unit connected to the special line unit, the oversized line dividing unit being used to re-divide the oversized line divided by the previous dividing unit, and the definition of the oversized line is that if the height of the line divided by the previous dividing unit is higher than a set multiple used for the oversized line of the standard height, the oversized line dividing unit is determined to be an oversized line; a space merging unit connected to the oversized line dividing unit, for receiving the handwritten manuscript processed by the oversized line dividing unit, for determining whether a new line is established after dividing the oversized line, and if not, for abandoning the division and accepting the original oversized line as one line, and for merging the lines formed by the isolated points and the overwritten strokes into other lines that are close to each other; a multi-line unit connected to the spatial merging unit and receiving the handwritten manuscript processed by the spatial merging unit, the multi-line unit calculating the distance between strokes in each of the lines that have already been divided, the distance being the difference between the rightmost stroke of the previous stroke and the leftmost stroke of the next stroke, and if the distance is greater than a set width for the lines, the multi-line unit using the previous stroke and the next stroke as a boundary to combine the previous and next strokes into two different lines, An information system based on automatic line breaks for handwritten documents, characterized in that line breaks are performed for handwritten manuscripts input based on the above method.

2. 2. The information system based on automatic line breaks for handwritten documents according to claim 1, further comprising a whole word determination unit connected to the line height unit, receiving the standard height and standard width calculated by the line height unit and determining whether the handwritten document is a single character, wherein the whole word determination unit calculates the height and width of the handwritten document, and recognizes the handwritten document as a single character if the height is smaller than a multiple of the standard height used for whole word determination, or if the width is smaller than a multiple of the standard width used for whole word determination, or if the aspect ratio is smaller than 2.

3. 2. An information system based on automatic line breaks for handwritten documents according to claim 1, wherein the line height unit arranges the strokes of a handwritten document based on the height of each stroke, and then sets the height of the strokes of a set proportional value as the standard height, and similarly arranges the strokes of all handwritten documents based on the stroke width, and then sets the width of the strokes of a set proportional value as the standard width.

4. 2. The information system based on automatic line breaks for handwritten documents according to claim 1, wherein the difference between the rightmost position of the previous line and the leftmost position of the next line is calculated as a horizontal axis difference dx, and the difference between the bottom of the first character of the previous line and the top of the first character of the next line is calculated as a height difference dy, wherein the horizontal axis difference dx must be greater than the set multiple of the standard width used for the previous division and the height difference dy must be greater than 0, and if the horizontal axis difference dx is greater than the set multiple of the standard width used for the previous division and the height difference dy is greater than 0, the two consecutive lines are divided; the clearer the deviation of the x-axis between the two consecutive lines, the more accurate the division; and if dx and dy do not satisfy the above conditions, the two consecutive lines are recognized as a single line.

5. The information system based on automatic line breaks for handwritten documents, as described in claim 1, characterized in that the definition of the isolated point is to obtain the y-axis distance Dy by subtracting the y-axis value of the midpoint of the y-axis direction of the stroke from the y-axis value of the point in the stroke, and in this manner, the y-axis distance is calculated for all points of all strokes in the handwritten manuscript, and these y-axis distances Dy are distributed. If all points of a stroke are outside two standard differences of this distribution, the stroke is considered to be an isolated point and the stroke is set to its own line.

6. The information system based on automatic line breaks for handwritten documents, as described in claim 1, characterized in that the inverted stroke is defined as a difference in the y-axis direction of the line to which the stroke and the currently written stroke belong when writing that is greater than a set multiple used to determine inverted strokes of the standard height, and this difference is set as the distance difference between the lowest point of the stroke on the y-axis and the highest point of the line to which the currently written stroke belongs.

7. 2. The information system based on automatic line wrapping for handwritten documents according to claim 1, wherein the oversized line dividing unit applies a clustering algorithm to obtain the coordinate axis of the height of each stroke, perform a clustering operation, classify each stroke based on its height, and then break each stroke based on the calculation result, thereby re-dividing the oversized line into multiple lines.

8. 2. An information system based on automatic line breaks for handwritten documents according to claim 1, wherein the oversized line dividing unit calculates the oversized line, then rearranges the divided lines divided from the oversized line and the original divided lines, and conforms the rearranged lines to the arrangement order of the original handwritten document.

9. 5. The information system based on automatic line breaks for handwritten documents according to claim 4, wherein the spatial merging unit calculates the horizontal axis difference dx and the height difference dy of the new line after dividing the oversized line, and determines whether such a line break is valid; if not, abandons the division and accepts the original oversized line as one line.

Images