Method, device and storage medium for constructing offline handwritten Uyghur database

Abstract

The invention discloses a method, device and storage medium for constructing an offline handwritten Uyghur database, characterized in that the construction method includes a corpus collection step and a corpus processing step; the corpus includes commonly used Uyghur handwritten characters; the construction method also includes correcting paragraph text and constructing line text after the corpus processing step; the corpus collection needs to be written and scanned in conjunction with an exclusive handwritten form; the corpus processing needs to correct and extract the form text; the invention realizes the construction of an offline handwritten Uyghur database quickly and systematically through a simple construction method, avoids the problems in the segmentation process such as incomplete segmentation through an exclusive handwritten form, provides a reference for other researchers through simple data collection, data processing, paragraph text correction and line text construction, and at the same time provides a research base for scientific research through the database, and greatly facilitates the lives of the Uyghur people.

Description

Technical Field

[0001] The invention relates to the technical field of text information processing, and in particular to a construction method and device for an offline handwritten Uighur database and a storage medium thereof. Background Art

[0002] Offline handwriting recognition is the process of obtaining text results on the image after transmitting a personal handwriting image to the machine. Offline handwriting recognition systems are widely used in actual work scenarios such as processing handwritten bills, handwritten test papers, and handwritten express delivery orders; therefore, the popularization of this technology can bring a lot of convenience when facing a large amount of handwritten text; offline handwriting databases are essential resources for the development of offline handwriting recognition systems, and the prerequisite for handwriting recognition is to have a database of the language, but currently there is still a lack of a Uyghur offline handwriting database in the known offline handwriting language databases. Because Uyghur offline handwriting databases face greater challenges when processing than printed text, a section of printed Uyghur text generally conforms to the regular arrangement pattern and has obvious dividing lines when dividing lines, so line division can be easily achieved through projection cutting; however, natural handwritten Uyghur text often has personal randomness, adhesion between lines, and collection of handwritten character labels, which will increase the difficulty of database construction, and Uyghur handwriting data set resources are scarce, and there is no complete construction method;

[0003] Based on this, the present invention designs a method and device for constructing an offline handwritten Uyghur database and a storage medium thereof to solve the above problems. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for constructing an offline handwritten Uyghur database and a storage medium thereof, which can effectively solve the problems raised in the above-mentioned background technology.

[0005] To solve the above problems, the technical solution adopted by the present invention is: a method, device and storage medium for constructing an offline handwritten Uyghur database, characterized in that the construction method includes a corpus collection step and a corpus processing step; the corpus contains commonly used Uyghur handwritten characters; the construction method also includes correcting paragraph text and constructing line text after the corpus processing step;

[0006] The corpus collection needs to be written and scanned in conjunction with a dedicated handwritten form;

[0007] The corpus processing needs to correct and extract the form text;

[0008] After the paragraph text is corrected, the text label of the paragraph text corresponds to the writing style on the exclusive handwriting form, and the text content of each line is consistent with the writing content of each line on the exclusive handwriting form;

[0009] After the line text is constructed, the line labels and line contents after the line text is segmented completely correspond to the writing style and writing contents on the exclusive handwriting form.

[0010] As a further preferred embodiment of the present invention, the step of collecting corpus includes:

[0011] A1. Use scripts to obtain news from the Uyghur section of Tianshan.com as a reference library for handwritten corpus;

[0012] A2. Create a dedicated handwriting form reference, and estimate the amount of text that each writer needs to write based on the dedicated handwriting form, so as to distribute the handwriting corpus;

[0013] A3. Scan and input the handwritten form completed by the writer;

[0014] The writers include people of all ages and educational backgrounds, and there are no restrictions on the writing style, dominant hand, and writing tools of the writers when filling out the form; the resolution of the image formed after scanning is 600dpi.

[0015] As a further preferred embodiment of the present invention, the correction of the form text comprises the following steps:

[0016] B1. Use polygon fitting algorithm to approximate the text rectangular area from outside to inside;

[0017] B2. After determining the area, generate the minimum circumscribed rectangle and obtain the coordinates of the four corners of the rectangle;

[0018] B3. Find the center coordinates, length, width and deflection angle of the rectangle through a set of coordinates. The deflection angle is θ. Rotate the image counterclockwise by θ° to obtain the corrected image.

[0019] B4. Affine transformation projects the content in the circumscribed rectangle onto an image with a resolution of 1400×1400. The above steps complete the image correction and extraction.

[0020] As a further preferred embodiment of the present invention, the affine transformation can be expressed as a 2x3 matrix H, which is composed of a set matrix A and a vector B; wherein:

[0021] The diagonal in A determines the scaling of the image, and the anti-diagonal determines the rotation;

[0022] B represents the translation of the image;

[0023]

[0024] As a further preferred solution of the present invention, the image obtained in step B4 is subjected to border elimination using a four-neighborhood seed filling algorithm in flood filling; after completion, a borderless image is obtained.

[0025] As a further preferred solution of the present invention, the borderless image is grayed out; and then Otsu binarization is used to obtain a borderless handwritten paragraph text image.

[0026] As a further preferred embodiment of the present invention, the annotator wraps and deletes the text labels of the borderless paragraph text image according to the writing style copied on the exclusive handwriting form to achieve line-by-line correspondence, thereby completing the correction of the paragraph text.

[0027] As a further preferred embodiment of the present invention, the line text construction includes line segmentation processing and line label correction; the line segmentation processing includes the following steps:

[0028] C1. Use the closing operation method in morphological image processing to obtain the ROI area;

[0029] C2, calculate the resolution of all ROI images and discard images with a resolution below 150,000;

[0030] C3, manually remove the images with segmentation errors and delete the images with segmentation errors;

[0031] C4, obtain the coordinates of all remaining ROI images and sort them according to the coordinates so that the output order is consistent with the original text;

[0032] The row label correction is to compare the image after paragraph text correction line by line after the row segmentation process is completed, and after verification with the segmentation result, delete the wrongly segmented image and the corresponding label to ensure that the data and the label are completely corresponding.

[0033] The present invention also relates to a device for constructing an offline handwritten Uyghur database, the device comprising a memory and a processor, the memory storing data, programs and instructions; the data, programs and instructions are loaded and executed by the processor to implement the operations performed by the method for constructing an offline handwritten Uyghur database as described above.

[0034] The present invention also relates to a readable storage medium, in which data, programs and instructions are stored; the data, programs and instructions are loaded and executed by the processor to implement the operations performed by the method for constructing an offline handwritten Uyghur database as described above.

[0035] Compared with the prior art, the present invention provides a method and device for constructing an offline handwritten Uyghur database and a storage medium thereof, which have the following beneficial effects:

[0036] This invention uses a simple construction method to quickly and systematically realize the construction of an offline handwritten Uyghur database, which fills some gaps in technological development and provides a great boost to my country's development; it avoids problems in the segmentation process such as incomplete segmentation through exclusive handwritten forms, and provides reference for other researchers through simple data collection, data processing, paragraph text correction, and line text construction. At the same time, the database provides a research foundation for scientific research and greatly facilitates the lives of the Uyghur people. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the present invention;

[0038] Figure 2 This is a flow chart of corpus collection of the present invention;

[0039] Figure 3 It is a flow chart of corpus processing of the present invention; DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0041] The present invention provides a method for constructing an offline handwritten Uighur database, characterized in that the method comprises the following steps:

[0042] 1. Data collection;

[0043] 1.1. A total of 421 articles were selected from the Uyghur version of Tianshan Net (http: / / uy.ts.cn / ), including 319 news articles published from February to August 2019 and 102 news articles published from April to December 2020, as database text tags, including domestic and foreign news, press conferences, latest meetings, the latest domestic laws, the latest policies, etc.

[0044] 1.2. Use a dedicated handwriting form to restrict the writing format, so that the difficulty in the segmentation and extraction steps is reduced, which will increase the convenience of subsequent work; the dedicated handwriting form includes a writing form and a spacing and marking form; the line spacing of the spacing and marking form is 1 to 3.5 cm; the standardized handwriting form provides higher collection efficiency for database construction; the dedicated handwriting form, through the combined use of the writing form and the spacing and marking form, has certain line spacing writing restrictions for the writer, which can improve subsequent work efficiency.

[0045] 1.3. The writer places the spacing mark form under the handwriting form and writes. After writing, it is scanned into an image with a resolution of 600dpi;

[0046] When filling out the form, there are no restrictions on the writing style or writing tools of the writers to ensure the randomness of the sample; and the writers include people of all ages and educational levels; more age groups and educational levels can bring more diverse handwriting styles; the test data set is more challenging; males account for 44.84% of the total writers and females account for 55.15%; while ensuring the randomness of the sample, the ratio of males to females is relatively balanced.

[0047] Uyghur is characterized by adhesion, polymorphism, and cursive writing. There are 34 characters in total, including 32 basic Uyghur characters and 2 special component characters. The basic type includes 8 vowels and 24 consonants. Letters have four forms: independent, front-linked, double-linked, and back-linked. There are a total of 128 letters through these different forms of permutations and combinations. When collecting corpus, it is necessary to include all or 90% of the commonly used Uyghur handwritten characters.

[0048] 2. Data processing includes:

[0049] 2.1. According to the rule, one side of the minimum circumscribed rectangle of a polygon must be collinear with one of the sides of the polygon.

[0050] 2.2. Traverse the edges of the polygon and fit the circumscribed rectangle.

[0051] 2.3. Compare the areas of all circumscribed rectangles and select the smallest one.

[0052] 2.4. Get the coordinate point set and rotation angle of the Box2D structure.

[0053] 2.5. Perform radial transformation according to the rotation angle and project it onto an image with a resolution of 1400×1400. Set the matrix A and vector B; where:

[0054] The diagonal in A determines the scaling of the image, and the anti-diagonal determines the rotation; B represents the translation of the image;

[0055] Affine transformation can be represented as a 2x3 matrix H, which consists of matrix A and vector B;

[0056]

[0057] 2.6. Use the four-neighborhood seed filling algorithm in flood filling to remove the border of the image after radial transformation projection; first give a seed point as the starting point, diffuse to the nearby adjacent pixel points, find all points with the same or similar colors, and fill them with new colors; repeat the above operation until no new pixels are filled, and these points will eventually form a connected area, thereby eliminating the image border and obtaining a borderless image;

[0058] 2.7. graying the borderless image, wherein the graying is performed by a weighted average method;

[0059] 2.8. Apply Otsu global threshold algorithm to the grayscaled borderless image; perform weighted averaging around it and set a reasonable threshold based on the characteristics of each image.

[0060] 3. Perform row segmentation processing;

[0061] 3.1. Use the closing operation method in morphological image processing to process the borderless image after binary processing; the connected domain image after the operation is the position of the text line to be extracted; return all connected domain images; select a 50×50 convolution kernel for the expansion part and a 10×10 convolution kernel for the erosion part; the number of erosion and expansion is 1;

[0062] 3.2. Limit the resolution of the connected domain when saving images and filter images that meet the conditions:

[0063] if(image_size<150000){delete}else{is ROI area};

[0064] The ROI image coordinates are set to (X i , Y i )(i=1,2,3……n);

[0065] 3.3. Sort the connected domain images according to the coordinates to ensure that the sequence number is consistent with the text order, that is:

[0066] Y i Sort in descending order to get the list H; save each group (X i , Y i ), obtain the ROI area in arrangement order, wherein the ROI area is a line segmentation image or a line text image.

[0067] 4. Label processing:

[0068] 4.1. Edit the text label to ensure that the text label is consistent with the content of each line of the handwritten image;

[0069] 4.2. After row segmentation, the handwritten image and label are compared row by row, and the incorrectly segmented images and corresponding labels are deleted to ensure that the data and labels are completely consistent.

[0070] Finally, we get a 1400×1400 pixel binary borderless handwritten document image, segmented text line images and their corresponding labels.

[0071] As a specific embodiment of the present invention:

[0072] First, the age, education and gender distribution of the writers are planned; then the self-designed form is used to collect the writers' handwriting data; the polygon fitting method is used to approximate the handwriting area to obtain the central coordinates and deflection angle of the area border, and the image is affine transformed according to the deflection angle to complete the image correction and extraction; the image border and noise are eliminated by the flood filling method to obtain a borderless paragraph image; the image is grayed and Otsu binarized, and then the morphological closing operation is used to obtain the ROI image after fixed threshold screening; the ROI image coordinate list is then sorted to obtain the final text line image; finally, the text label is edited to ensure that the text label is consistent with the content of each line of the handwritten image; after line segmentation, the existing image and label are compared line by line, and the wrongly segmented images and corresponding labels are deleted to ensure that the data and labels are completely corresponding.

[0073] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing an offline handwritten Uyghur database, characterized in that: The construction method includes the steps of corpus collection and corpus processing; the corpus includes commonly used Uyghur handwritten characters; the construction method also includes correcting paragraph text and constructing line text after the corpus processing step; The corpus collection needs to be written and scanned in conjunction with a dedicated handwritten form; The corpus processing needs to correct and extract the form text; After the paragraph text is corrected, the text label of the paragraph text corresponds to the writing style on the exclusive handwriting form, and the text content of each line is consistent with the writing content of each line on the exclusive handwriting form; After the line text is constructed, the line labels and line contents after the line text is segmented completely correspond to the writing style and writing content on the exclusive handwriting form; The steps of corpus collection include: A1. Use scripts to obtain news from the Uyghur section of Tianshan.com as a reference library for handwritten corpus; A2. Create a dedicated handwriting form reference, and estimate the amount of text that each writer needs to write based on the dedicated handwriting form, so as to distribute the handwriting corpus; A3. Scan and input the handwritten form completed by the writer; The writers include people of various ages and educational backgrounds, and the writing style, dominant hand, and writing tools of the writers during the filling of the form; the resolution of the image formed after scanning is 600dpi; The correction of the form text comprises the following steps: B1. Use polygon fitting algorithm to approximate the text rectangular area from outside to inside; B2. After determining the area, generate the minimum circumscribed rectangle and obtain the coordinates of the four corners of the rectangle; B3. Find the center coordinates, length, width and deflection angle of the rectangle through a set of coordinates. The deflection angle is θ. Rotate the image counterclockwise by θ° to obtain the corrected image. B4. Affine transformation projects the content in the circumscribed rectangle onto an image with a resolution of 1400×1400. The image correction and extraction are completed through the above steps. The affine transformation can be expressed as a 2x3 matrix H, which is composed of a set matrix A and a vector B; wherein: The diagonal in A determines the scaling of the image, and the anti-diagonal determines the rotation; B represents the translation of the image; 2. The method for constructing an offline handwritten Uyghur database according to claim 1, characterized in that: The image obtained in step B4 is subjected to border elimination using the four-neighborhood seed filling algorithm in flood filling; after completion, a borderless image is obtained.

3. The method for constructing an offline handwritten Uyghur database according to claim 2, characterized in that: The borderless image is grayed out; and then Otsu binarization is performed to obtain a borderless handwritten paragraph text image.

4. The method for constructing an offline handwritten Uyghur database according to claim 1 or 3, characterized in that: The annotator wraps and deletes the text labels of the borderless paragraph text image according to the writing style copied on the exclusive handwriting form to achieve line-by-line correspondence, and completes the correction of the paragraph text.

5. The method for constructing an offline handwritten Uyghur database according to claim 1, characterized in that: The line text construction includes line segmentation processing and line label correction; the line segmentation processing includes the following steps: C1. Use the closing operation method in morphological image processing to obtain the ROI area; C2, calculate the resolution of all ROI images and discard images with a resolution below 150,000; C3, manually remove the images with segmentation errors and delete the images with segmentation errors; C4, obtain the coordinates of all remaining ROI images and sort them according to the coordinates so that the output order is consistent with the original text; The row label correction is to compare the image after paragraph text correction row by row after the row segmentation process is completed, and after verification with the segmentation result, delete the wrongly segmented image and the corresponding label to ensure that the data and the label are completely corresponding.

6. A device for constructing an offline handwritten Uyghur database, characterized in that: The device includes a memory and a processor, wherein the memory stores data, programs and instructions; the data, programs and instructions are loaded and executed by the processor to implement the operations performed by the method for constructing an offline handwritten Uyghur database as described in any one of claims 1 to 5.

7. A readable storage medium, characterized in that: The storage medium stores data, programs and instructions; the data, programs and instructions are loaded and executed by the processor to implement the operations performed by the method for constructing an offline handwritten Uyghur database as described in any one of claims 1 to 5.

Images