A method and system for stroke order recognition of handwritten Chinese character images

Through Chinese character skeleton extraction and eight-neighborhood direction coding rules, combined with branch boundary method and centroid distance, the difficulty of stroke order recognition in static handwritten Chinese characters images is solved, improving the recognition accuracy and improving the writing experience.

CN113903045BActive Publication Date: 2024-11-01HUAZHONG NORMAL UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202111237628.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-22
Publication Date
2024-11-01
Estimated Expiration
2041-10-22

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify stroke orders in static handwritten Chinese characters images, especially in the absence of time sequence data in static images, which leads to difficulty in identifying stroke orders and distortion in writing experience of electronic devices.

Method used

Through the extraction and segmentation of Chinese characters, combined with the eight-neighborhood direction coding rules, the branch boundary method is used to match stroke orders and fusion of stroke segments, and the accurate matching and fusion of stroke segments is achieved using the center of mass distance and legal direction coding.

Benefits of technology

It improves the accuracy of stroke order recognition in static handwritten Chinese characters images, provides a more realistic writing experience, avoids distortion of the writing experience of electronic devices, and achieves detailed matching and fusion of different stroke types.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113903045B_ABST
    Figure CN113903045B_ABST
Patent Text Reader

Abstract

The invention discloses a stroke order recognition method and system for handwritten Chinese character images, wherein the stroke order recognition method comprises a preprocessing step, a first stroke order matching step, a second stroke order matching step and a stroke segment fusion step; the stroke order recognition method provided by the invention is for static handwritten Chinese character images, and avoids distortion of writing experience on electronic devices; the invention provides illegal direction codes of different types of strokes based on eight-neighborhood direction coding rules, and calculates matching weights in the first stroke order matching step according to the illegal direction codes; the invention performs the first stroke order matching and the second stroke order matching on handwritten Chinese characters, and can ensure that each stroke segment in the handwritten Chinese characters has a matching stroke order; the invention realizes stroke segment fusion through eight-neighborhood coding chains of different stroke segments and legal direction coding of different strokes; the stroke order recognition method provided by the invention can effectively improve the accuracy of stroke order recognition in static handwritten Chinese character images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and more specifically, relates to a stroke order recognition method and system for handwritten Chinese character images. Background Art

[0002] In the entire process of standardizing Chinese character writing teaching, Chinese character teaching in the lower grades of primary schools is the foundation, but most learners in the lower grades write Chinese characters in a variety of styles, and it is difficult to provide correction information and assist in correction for all the Chinese characters written by each student. In addition, most parents and teachers have not received strict and systematic education on standardizing Chinese character writing. Even if one-on-one tutoring can be achieved in the end, teachers and parents cannot provide specific and standardized evaluation information on the Chinese characters written by students.

[0003] With the continuous development and improvement of information technology, many computer-assisted Chinese character writing evaluation systems have emerged. Related research work can be divided into two categories: post-evaluation and real-time evaluation. Post-evaluation means that the writer is allowed to write the target Chinese character in one go without being disturbed, and then the feature data of the handwritten Chinese character is extracted and compared with the template Chinese character for normative evaluation; real-time evaluation means that every time the writer finishes writing a stroke of the target Chinese character, the system immediately judges its normativeness. There have been considerable achievements in the research of these two types of systems, and the main focus is on the correctness of the stroke order of handwritten Chinese characters.

[0004] To deeply study the correctness evaluation of the stroke order of handwritten Chinese characters, it is very important to digitize handwritten Chinese characters. At present, many Chinese character writing teaching systems collect Chinese character data through devices such as writing tablets and capacitive pens. Electronic devices have time series data and can easily collect the stroke order information of the writer's handwritten Chinese characters. However, most of these devices are relatively hard and have smooth surfaces, which affects the writing experience. It is difficult to get the real writing experience of pen and paper when writing on an electronic screen. In addition, after the electronic device collects the Chinese character segment data, it generally sets a threshold to achieve segment fusion, and rarely considers the characteristics of the Chinese character strokes themselves.

[0005] Let the writer use a hard pen to write the target Chinese characters on a specific paper, collect the original image by taking a photo, and finally extract the handwriting data of the handwritten Chinese characters from it through computer image processing technology. This can avoid the distortion of the writer's writing experience, but it also brings difficulties in stroke order recognition. Static handwritten Chinese character images have no time series data, and there are many difficulties in recognizing the stroke order of handwritten Chinese characters from static handwritten Chinese character images.

[0006] The Chinese character skeleton extraction operation is an important step in the stroke order recognition of static handwritten Chinese characters. Performing the Chinese character skeleton extraction operation on static handwritten Chinese characters can obtain the Chinese character skeleton, which can correctly reflect the topological structure of the Chinese character without unnecessary burrs or side branches.

[0007] Any pixel P in a static handwritten Chinese character image has 4 adjacent pixels, located above, below, left and right of it respectively. These 4 pixels are the 4-neighborhood of pixel P; the D-neighborhood of pixel P is the points corresponding to the four vertices of pixel p; the 4-neighborhood and D-neighborhood of pixel P constitute the 8-neighborhood of pixel P; the directional encoding rule of the 8-neighborhood is: starting from the neighborhood point on the left of pixel P, the 8-neighborhood is marked as P0, P1, P2, P3, P4, P5, P6, and P7 counterclockwise; the direction from P to P0 is encoded as 0, the direction from P to P1 is encoded as 1, the direction from P to P2 is encoded as 2, the direction from P to P3 is encoded as 3, the direction from P to P4 is encoded as 4, the direction from P to P5 is encoded as 5, the direction from P to P6 is encoded as 6, and the direction from P to P7 is encoded as 7.

[0008] The pixels that make up the handwritten Chinese character in a static handwritten Chinese character image are handwriting pixels. Deleting the intersections in the Chinese character skeleton can get multiple segments, each of which is composed of multiple handwriting pixels. When all the handwriting pixels in a segment are traversed, the direction coding of a handwriting pixel A is recorded as follows:

[0009] (1) Determine the previous stroke pixel point A0 of A in the traversal operation;

[0010] (2) Find the position of A in the eight neighborhoods of A0 and the direction code X from A0 to A;

[0011] (3) The direction code of A is recorded as X, that is, the handwriting pixel point corresponding to X is A.

[0012] Therefore, the first handwriting pixel point accessed by the traversal operation has no direction code, and the direction codes of other handwriting pixels in the segment are combined into a set, which is the eight-neighborhood code chain of the segment.

[0013] When establishing an eight-neighborhood coding chain for the standard Chinese characters in the national standard first-level character library according to the eight-neighborhood directional coding rules, the handwriting pixel points in the above recording method are replaced with the pixel points that constitute the standard Chinese characters, and the stroke segments are replaced with strokes, so that the eight-neighborhood coding chain of each stroke that constitutes the standard Chinese characters can be obtained.

[0014] In a static handwritten Chinese character image or a standard Chinese character image, the upper left corner of the image is used as the origin of the coordinate system, the horizontal rightward direction is the positive direction of the horizontal coordinate axis, and the vertical downward direction is the positive direction of the vertical coordinate axis. Selecting this coordinate system can make the specific coordinate data all positive integers.

[0015] The centroid of a static handwritten Chinese character segment or a standard Chinese character stroke is the average value of all pixel coordinate values ​​of the segment or the stroke. Summary of the invention

[0016] In view of the defects of the prior art, the purpose of the present invention is to provide a method and system for stroke order recognition of handwritten Chinese character images, aiming to solve the problem of stroke order recognition in static handwritten Chinese character images.

[0017] To achieve the above object, the present invention provides a method for stroke order recognition of a handwritten Chinese character image, comprising the following steps:

[0018] (1) Preprocessing step: performing a Chinese character skeleton extraction operation on the original handwritten Chinese character image to obtain a Chinese character skeleton image; performing a segmentation operation on the Chinese character skeleton to obtain a segmented image containing multiple stroke segments; the original handwritten Chinese character image only contains one handwritten Chinese character;

[0019] (2) a first stroke order matching step: performing a stroke order matching operation on the stroke segments in the segmented image according to the stroke set by using the branch and bound method, forming a successful stroke segment set with the successfully matched stroke segments, and completing a first mapping from the successful stroke segment set to the stroke set: mapping each stroke segment to a successfully matched stroke order; forming a failed stroke segment set with the failed stroke segments, and performing the following second stroke order matching step on the failed stroke segment set;

[0020] The method for obtaining the stroke set is: searching in a standard Chinese character library to obtain a standard stroke set of the handwritten Chinese characters in the original handwritten Chinese character image, referred to as the stroke set;

[0021] The method for establishing the standardized Chinese character library is as follows: establishing a standardized Chinese character library for all Chinese characters in the national standard first-level character library according to the eight-neighborhood direction encoding rule, wherein the standardized Chinese character library is composed of a standard stroke set of each Chinese character, and the standard stroke set includes the stroke order and centroid of each stroke;

[0022] (3) A second stroke order matching step: for each stroke segment in the failed stroke segment set, respectively calculating the distance between the centroid of the stroke segment and the centroid of each stroke in the stroke set, and forming a distance set; finding the minimum distance in the distance set; finding the stroke order of the stroke corresponding to the minimum distance in the stroke set, and determining that the stroke order successfully matches the stroke segment; completing a second mapping from the failed stroke segment set to the stroke set: mapping each stroke segment to a successfully matched stroke order;

[0023] (4) Segment fusion step: merging the first mapping and the second mapping into a third mapping, and forming a segment set to be merged of the stroke order with two or more segments mapped to the same stroke order in the third mapping; performing a segment fusion operation on each of the segment sets to be merged: merging the segments in each of the segment sets to be merged into one segment.

[0024] Preferably, any handwriting pixel point of the Chinese character skeleton image in the preprocessing step has no more than 4 adjacent handwriting pixel points in its eight neighborhoods; the starting and ending points of the stroke segment in the preprocessing step have no more than 1 adjacent handwriting pixel point in its eight neighborhoods, and other handwriting pixel points have no more than 2 adjacent handwriting pixel points in their eight neighborhoods;

[0025] The standard stroke set in the standard Chinese character library also includes illegal direction codes and legal direction codes for each stroke;

[0026] The rules for illegal direction encoding are:

[0027] If the main direction of the stroke is coded as 0, the illegal directions are coded as 2 and 6;

[0028] If the main direction of the stroke is coded as 1, the illegal directions are coded as 3 and 7;

[0029] If the main direction of the stroke is coded as 2, the illegal directions are coded as 0 and 4;

[0030] If the main direction of the stroke is coded as 3, the illegal directions are coded as 1 and 5;

[0031] If the main direction of the stroke is coded as 4, the illegal directions are coded as 2 and 6;

[0032] If the main direction of the stroke is coded as 5, the illegal directions are coded as 3 and 7;

[0033] If the main direction of the stroke is coded as 6, the illegal directions are coded as 0 and 4;

[0034] If the main direction of the stroke is coded as 7, the illegal directions are coded as 1 and 5;

[0035] The rules for legal direction encoding are:

[0036] If the main direction of the stroke is coded as 0, then the legal direction codes are 0, 1, and 7;

[0037] If the main direction of the stroke is coded as 1, then the legal directions are coded as 0, 1, and 2;

[0038] If the main direction of the stroke is coded as 2, then the legal directions are coded as 1, 2, and 3;

[0039] If the main direction of the stroke is coded as 3, then the legal directions are coded as 2, 3, and 4;

[0040] If the main direction of the stroke is coded as 4, then the legal directions are coded as 3, 4, and 5;

[0041] If the main direction of the stroke is coded as 5, then the legal directions are coded as 4, 5, and 6;

[0042] If the main direction of the stroke is coded as 6, then the legal directions are coded as 5, 6, and 7;

[0043] If the main direction of the stroke is coded as 7, then the legal direction codes are 0, 6, and 7;

[0044] The main direction code of the stroke is the direction code that appears most frequently in the eight-neighborhood code chain of the stroke.

[0045] Preferably, the stroke segment matching operation in the first stroke order matching step includes the following steps:

[0046] (1) Matching weight calculation step: Calculate the main direction code of each of the stroke segments, and compare the illegal direction codes of all the strokes in the stroke set respectively: If the main direction code of the stroke segment does not belong to the illegal direction code of one of the strokes, calculate the matching weight of the stroke segment and the stroke, and continue to compare with the next stroke; otherwise, determine that the stroke segment does not match the stroke, and continue to compare with the next stroke; until all strokes are compared;

[0047] The main direction code of the segment is the direction code that appears most frequently in the eight-neighborhood code chain of the segment;

[0048] The matching weight is the distance between the centroid of the stroke segment and the centroid of the stroke;

[0049] (2) Stroke order matching step: According to all matching weights of each stroke segment, the first mapping is completed by the branch and bound method.

[0050] Preferably, the segment fusion operation in the segment fusion step includes the following steps:

[0051] (1) Segment coding step: starting from the starting point, a traversal operation is performed on each of the stroke segments in the to-be-fused stroke segment set, and the direction coding of each stroke pixel except the starting point is calculated according to the eight-neighborhood direction coding rule, and each of the direction codes forms an eight-neighborhood coding chain of the stroke segment in the traversal order of the traversal operation;

[0052] (2) Flipping a segment step: for each segment in the set of segments to be merged, search for the image of the segment in the third mapping, and search for the legal direction code of the corresponding stroke in the stroke set according to the image of the segment; determine whether the eight-neighborhood code chain of the segment is included in the legal direction code; if not, flip the segment, put the flipped segment into the set of segments to be merged, and delete the segment before flipping; otherwise, do not flip the segment, and keep it in the set of segments to be merged;

[0053] (3) Segment fusion step: randomly select two segments from the set of segments to be fused: the first segment and the second segment, and calculate the distances D1, D2, D3 and D4 for the starting pixel point P1 and the ending pixel point P2 of the first segment, and the starting pixel point P3 and the ending pixel point P4 of the second segment; D1 is the distance between P1 and P3, D2 is the distance between P1 and P4, D3 is the distance between P2 and P3, and D4 is the distance between P2 and P4; if the smallest distance among D1 to D4 is less than a set threshold, the two points with the closest distance among P1 to P4 are selected for fusion, and the fused segment is placed in the set of segments to be fused; otherwise, the fusion is determined to have failed, and the segment that failed to be fused is deleted; the segment that failed to be fused is the segment in the first segment and the second segment that belongs to the set of failed segments;

[0054] (4) Fusion completion step: Repeat the fusion step for the segment set to be fused until it is fused into one segment.

[0055] The present invention provides a stroke order recognition system for handwritten Chinese character images, comprising:

[0056] A stroke set acquisition module is used to establish a standardized Chinese character library for all Chinese characters in the national standard first-level character library according to the eight-neighborhood direction encoding rule, wherein the standardized Chinese character library is composed of a standard stroke set of each Chinese character, and the standard stroke set includes the stroke order and centroid of each stroke; and to search the standard stroke set of the handwritten Chinese characters in the original handwritten Chinese character image in the standardized Chinese character library, hereinafter referred to as the stroke set;

[0057] The original handwritten Chinese character image contains only one handwritten Chinese character;

[0058] Preprocessing module: used for performing Chinese character skeleton extraction operation on the original handwritten Chinese character image to obtain a Chinese character skeleton image; performing segmentation operation on the Chinese character skeleton to obtain a segmented image containing multiple stroke segments;

[0059] The first stroke order matching module is used to perform stroke order matching operations on the stroke segments in the segmented image according to the stroke set by using the branch and bound method, and the successfully matched stroke segments are grouped into a successful stroke segment set, and the first mapping from the successful stroke segment set to the stroke set is completed: each stroke segment is mapped to a successfully matched stroke order; the unmatched stroke segments are grouped into a failed stroke segment set, and the operations in the second stroke order matching module are performed on the failed stroke segment set;

[0060] The second stroke order matching module is used to calculate the distance between the centroid of each stroke in the stroke set and the centroid of each stroke in the stroke set for each stroke in the failed stroke set, and form a distance set; find the minimum distance in the distance set; find the stroke order of the stroke corresponding to the minimum distance in the stroke set, and determine that the stroke order matches the stroke set successfully; complete the second mapping from the failed stroke set to the stroke set: map each stroke to the stroke order of successful matching;

[0061] The stroke segment fusion module is used to merge the first mapping and the second mapping into a third mapping, and form a stroke segment set to be merged of the stroke order with more than two stroke segments mapped to the same stroke order in the third mapping; and perform a stroke segment fusion operation on each of the stroke segment sets to be merged respectively: merge the stroke segments in each of the stroke segment sets to be merged into one stroke segment.

[0062] Preferably, any handwriting pixel point of the Chinese character skeleton image in the preprocessing module has no more than 4 adjacent handwriting pixel points in its eight neighborhoods; the starting and ending points of the stroke segments in the preprocessing module have no more than 1 adjacent handwriting pixel point in its eight neighborhoods, and other handwriting pixel points have no more than 2 adjacent handwriting pixel points in their eight neighborhoods;

[0063] The standard stroke set in the stroke set acquisition module also includes an illegal direction code and a legal direction code of each stroke;

[0064] The rules for illegal direction encoding are:

[0065] If the main direction of the stroke is coded as 0, the illegal directions are coded as 2 and 6;

[0066] If the main direction of the stroke is coded as 1, the illegal directions are coded as 3 and 7;

[0067] If the main direction of the stroke is coded as 2, the illegal directions are coded as 0 and 4;

[0068] If the main direction of the stroke is coded as 3, the illegal directions are coded as 1 and 5;

[0069] If the main direction of the stroke is coded as 4, the illegal directions are coded as 2 and 6;

[0070] If the main direction of the stroke is coded as 5, the illegal directions are coded as 3 and 7;

[0071] If the main direction of the stroke is coded as 6, the illegal directions are coded as 0 and 4;

[0072] If the main direction of the stroke is coded as 7, the illegal directions are coded as 1 and 5;

[0073] The rules for legal direction encoding are:

[0074] If the main direction of the stroke is coded as 0, then the legal direction codes are 0, 1, and 7;

[0075] If the main direction of the stroke is coded as 1, then the legal directions are coded as 0, 1, and 2;

[0076] If the main direction of the stroke is coded as 2, then the legal directions are coded as 1, 2, and 3;

[0077] If the main direction of the stroke is coded as 3, then the legal directions are coded as 2, 3, and 4;

[0078] If the main direction of the stroke is coded as 4, then the legal directions are coded as 3, 4, and 5;

[0079] If the main direction of the stroke is coded as 5, then the legal directions are coded as 4, 5, and 6;

[0080] If the main direction of the stroke is coded as 6, then the legal directions are coded as 5, 6, and 7;

[0081] If the main direction of the stroke is coded as 7, then the legal direction codes are 0, 6, and 7;

[0082] The main direction code of the stroke is the direction code that appears most frequently in the eight-neighborhood code chain of the stroke.

[0083] Preferably, the first stroke order matching module includes the following submodules:

[0084] A matching weight calculation submodule is used to calculate the main direction code of each of the stroke segments, and compare the illegal direction codes of all the strokes in the stroke set respectively: if the main direction code of the stroke segment does not belong to the illegal direction code of one of the strokes, then the matching weight of the stroke segment and the stroke is calculated, and the comparison is continued with the next stroke; otherwise, it is determined that the stroke segment does not match the stroke, and the comparison is continued with the next stroke; until the comparison of all strokes is completed;

[0085] The main direction code of the segment is the direction code that appears most frequently in the eight-neighborhood code chain of the segment;

[0086] The matching weight is the distance between the centroid of the stroke segment and the centroid of the stroke;

[0087] Matching stroke order submodule: used to complete the first mapping through the branch and bound method according to all matching weights of each stroke segment.

[0088] Preferably, the stroke fusion module includes the following submodules:

[0089] The stroke segment coding submodule is used to perform a traversal operation on each stroke segment in the stroke segment set to be fused, starting from the starting point, and calculating the direction code of each stroke pixel point except the starting point according to the eight-neighborhood direction coding rule. Each of the direction codes constitutes an eight-neighborhood coding chain of the stroke segment in the traversal order of the traversal operation;

[0090] The flipping segment submodule is used to search the image of each segment in the set of segments to be merged in the third mapping, and search the legal direction code of the corresponding stroke in the stroke set according to the image of the segment; determine whether the eight-neighborhood code chain of the segment is included in the legal direction code, and if not, flip the segment, put the flipped segment into the set of segments to be merged, and delete the segment before flipping; otherwise, do not flip the segment, and keep it in the set of segments to be merged;

[0091] The fusion segment submodule is used to select any two segments from the set of segments to be fused: the first segment and the second segment, and calculate the distances D1, D2, D3 and D4 for the starting pixel point P1 and the ending pixel point P2 of the first segment, and the starting pixel point P3 and the ending pixel point P4 of the second segment; D1 is the distance between P1 and P3, D2 is the distance between P1 and P4, D3 is the distance between P2 and P3, and D4 is the distance between P2 and P4; if the smallest distance among D1 to D4 is less than the set threshold, the two points with the closest distance among P1 to P4 are taken for fusion, and the fused segment is put into the set of segments to be fused; otherwise, it is determined that the fusion fails, and the segment that fails to be fused is deleted; the segment that fails to be fused is the segment in the first segment and the second segment that belongs to the set of failed segments;

[0092] Fusion completion submodule: used to repeatedly execute the operations in the fusion segment submodule on the fused segment set until it is fused into one segment.

[0093] The present invention provides a stroke order recognition device for handwritten Chinese character images, characterized in that it includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement the stroke order recognition method for handwritten Chinese character images as described above when executing the computer program.

[0094] The present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and when the computer program is executed by a processor, the stroke order recognition method of handwritten Chinese character images as described above is implemented.

[0095] Through the above technical scheme conceived by the present invention, compared with the prior art, since the stroke order recognition method provided by the present invention is for static handwritten Chinese character images, the writing experience of the writer when writing Chinese characters on paper is more realistic and smoother, avoiding the distortion of the writer's writing experience on electronic devices.

[0096] The present invention fully considers the different stroke types of Chinese characters, and studies and summarizes this based on the eight-neighborhood direction coding rule, gives illegal direction coding of different types of strokes, and calculates the matching weight in the first stroke order matching step according to the illegal direction coding. The first stroke order matching method of the present invention is detailed and comprehensive, and can achieve the beneficial effect of high stroke order matching accuracy.

[0097] The matching weight of the present invention is the distance between the centroid of the stroke segment and the centroid of the stroke, and the calculation method is simple and effective.

[0098] The present invention performs the first stroke order matching and the second stroke order matching on the handwritten Chinese characters, thereby ensuring that each stroke segment in the handwritten Chinese characters has a matching stroke order.

[0099] Compared with the prior art that realizes stroke segment fusion by setting thresholds, the present invention fully considers the different stroke types of Chinese characters and realizes stroke segment fusion through the eight-neighborhood coding chains of different strokes and the legal direction coding of different strokes. The stroke segment fusion method of the present invention is more detailed and comprehensive, and can achieve the beneficial effect of high stroke segment fusion accuracy.

[0100] The stroke order recognition method provided by the present invention can effectively improve the accuracy of stroke order recognition in static handwritten Chinese character images. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 is a flow chart of a method for stroke order recognition of a handwritten Chinese character image provided by an embodiment of the present invention;

[0102] Figure 2 It is a flow chart of the stroke order matching operation in the first stroke order matching step in the stroke order recognition method for handwritten Chinese character images provided by an embodiment of the present invention;

[0103] Figure 3 It is a flow chart of the stroke segment fusion operation in the stroke segment fusion step in the stroke order recognition method for handwritten Chinese character images provided by an embodiment of the present invention;

[0104] Figure 4 is an original handwritten Chinese character image in the stroke order recognition method for a handwritten Chinese character image provided by an embodiment of the present invention;

[0105] Figure 5 It is a marked Chinese character image in the stroke order recognition method of a handwritten Chinese character image provided by an embodiment of the present invention;

[0106] Figure 6It is a schematic diagram of eight-neighborhood direction coding rules in the stroke order recognition method of handwritten Chinese character images provided by an embodiment of the present invention;

[0107] Figure 7 It is a schematic diagram of segmenting stroke segments in an image in a method for stroke order recognition of a handwritten Chinese character image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0108] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0109] like Figure 1 As shown, the stroke order recognition method for handwritten Chinese character images provided by the embodiment of the present invention includes a preprocessing step, a first stroke order matching step, a second stroke order matching step, a stroke segment fusion step and a restoration and normalization step.

[0110] First, the preprocessing steps are performed: the Chinese character skeleton is extracted from the original handwritten Chinese character image to obtain a Chinese character skeleton image; and the Chinese character skeleton is segmented to obtain a segmented image containing multiple strokes.

[0111] The original handwritten Chinese character image is as follows Figure 4 As shown, the method for obtaining the original handwritten Chinese character image includes the following steps:

[0112] A0: Design a grid of uniform size on A4 size paper;

[0113] A1: Please write a Chinese character in each square;

[0114] A2: Use a camera to take a picture of the written paper to obtain the original image;

[0115] A3: Extract the contour of the original image, segment each square, and extract the handwritten Chinese characters in each square; the handwritten Chinese character image in a single square is the original handwritten Chinese character image.

[0116] The stroke order recognition method provided by the embodiment of the present invention is for static handwritten Chinese character images, and the writing experience of the writer when writing Chinese characters on paper is more realistic and smoother, thereby avoiding the distortion of the writer's writing experience on an electronic device.

[0117] The Chinese character skeleton extraction operation includes the following steps:

[0118] B0: Perform normalization processing on the original handwritten Chinese character image, including the following steps:

[0119] B0-0: Perform contour recognition on a single Chinese character image and extract the enclosing rectangle of the Chinese character;

[0120] B0-1: Expand and extract the square bounding box from the bounding rectangle;

[0121] B0-2: Use the perspective projection matrix to extract the square bounding box and scale it to a size of 500*500 pixels.

[0122] B1: Performing a thinning operation on the normalized Chinese character image to obtain a Chinese character skeleton image, including the following steps:

[0123] B1-0: grayscale and binarization processing operations are performed on the Chinese character image in sequence;

[0124] B1-1: Use the Zhang-Suen thinning algorithm to process the Chinese character image and obtain a preliminary thinned Chinese character skeleton;

[0125] B1-2: Use the index table refinement algorithm to process the initially refined Chinese character skeleton again to obtain the final Chinese character skeleton image. Any handwriting pixel point in the Chinese character skeleton image has no more than 4 adjacent handwriting pixels in its eight neighborhoods.

[0126] The split operation includes the following steps:

[0127] C0: Delete the intersection points in the Chinese character skeleton image, store the intersection points in the first list L1, delete the redundant noise points, divide the Chinese character strokes into multiple stroke segments, and obtain a preliminary segmented image; among which, the handwriting pixel points whose number of handwriting pixel points contained in their eight neighborhoods is greater than or equal to 3 are intersection points;

[0128] C1: traverse the initial segmented image, find the starting and ending points of all the strokes, and store them in the second list L2;

[0129] C2: Take out the starting and ending points of each segment from the second list L2, traverse the segment, find the inflection point in the segment, and delete the inflection point;

[0130] C3: traverse the first list L1 of intersection points, connect two intersection points whose distance is less than the threshold, if the connected segment has no inflection point, keep the connected segment; if there is an inflection point, cancel the connection; the threshold can be set to 6.

[0131] like Figure 7 As shown, after the segmentation operation is completed, the first starting and ending points L and the second starting and ending points M of the pen segment have only one adjacent handwriting pixel point in their eight neighborhoods, and the other handwriting pixels have only two adjacent handwriting pixels in their eight neighborhoods.

[0132] The first stroke order matching step is: using the branch and bound method, the stroke order matching operation is performed on the stroke segments in the segmented image according to the stroke set, the successfully matched stroke segments are grouped into a successful stroke segment set, and the first mapping from the successful stroke segment set to the stroke set is completed: each stroke segment is mapped to a successfully matched stroke order; the unmatched stroke segments are grouped into a failed stroke segment set, and the following second stroke order matching step is performed on the failed stroke segment set.

[0133] The method for obtaining the stroke set is: searching the standard stroke set of the handwritten Chinese characters in the original handwritten Chinese character image in the standard Chinese character library, which is referred to as the stroke set for short.

[0134] The method for establishing a standardized Chinese character library is as follows: Figure 6 The eight-neighborhood direction encoding rules shown establish a standardized Chinese character library for all Chinese characters in the national standard first-level character library. The standardized Chinese character library is composed of a standard stroke set for each Chinese character. The standard stroke set includes the stroke order, centroid, illegal direction code and legal direction code of each stroke.

[0135] The rules for illegal direction encoding are:

[0136] If the main direction of the stroke is coded as 0, the illegal directions are coded as 2 and 6;

[0137] If the main direction of the stroke is coded as 1, the illegal directions are coded as 3 and 7;

[0138] If the main direction of the stroke is coded as 2, the illegal directions are coded as 0 and 4;

[0139] If the main direction of the stroke is coded as 3, the illegal directions are coded as 1 and 5;

[0140] If the main direction of the stroke is coded as 4, the illegal directions are coded as 2 and 6;

[0141] If the main direction of the stroke is coded as 5, the illegal directions are coded as 3 and 7;

[0142] If the main direction of the stroke is coded as 6, the illegal directions are coded as 0 and 4;

[0143] If the main direction of the stroke is coded as 7, the illegal directions are coded as 1 and 5;

[0144] The rules for legal direction encoding are:

[0145] If the main direction of the stroke is coded as 0, then the legal direction codes are 0, 1, and 7;

[0146] If the main direction of the stroke is coded as 1, then the legal directions are coded as 0, 1, and 2;

[0147] If the main direction of the stroke is coded as 2, then the legal directions are coded as 1, 2, and 3;

[0148] If the main direction of the stroke is coded as 3, then the legal directions are coded as 2, 3, and 4;

[0149] If the main direction of the stroke is coded as 4, then the legal directions are coded as 3, 4, and 5;

[0150] If the main direction of the stroke is coded as 5, then the legal directions are coded as 4, 5, and 6;

[0151] If the main direction of the stroke is coded as 6, then the legal directions are coded as 5, 6, and 7;

[0152] If the main direction of the stroke is coded as 7, then the legal direction codes are 0, 6, and 7;

[0153] The main direction code of a stroke is the direction code that appears most frequently in the eight-neighborhood code chain of the stroke.

[0154] like Figure 2 As shown, the stroke order matching operation includes a matching weight calculation step and a stroke order matching step:

[0155] Matching weight calculation steps: calculate the main direction code of each stroke segment, and compare the illegal direction codes of all strokes in the stroke set respectively: if the main direction code of the stroke segment does not belong to the illegal direction code of one of the strokes, calculate the matching weight of the stroke segment and the stroke, and continue to compare the next stroke; otherwise, determine that the stroke segment does not match the stroke, and continue to compare the next stroke; until all strokes are compared;

[0156] The main direction code of a segment is the direction code that appears most frequently in the eight-neighborhood code chain of the segment.

[0157] The matching weight is the distance between the centroid of the stroke segment and the centroid of the stroke. The matching weight calculation method of the embodiment of the present invention is simple and effective.

[0158] The embodiment of the present invention fully considers the different stroke types of Chinese characters, and studies and summarizes this based on the eight-neighborhood direction coding rules, gives illegal direction coding of different types of strokes, and calculates the matching weight in the first stroke order matching step based on the illegal direction coding. The first stroke order matching method of the embodiment of the present invention is detailed and comprehensive, and can achieve the beneficial effect of high stroke order matching accuracy.

[0159] Matching stroke order step: According to all matching weights of each stroke segment, stroke order matching is performed by the branch and bound method, and the successfully matched stroke segments are grouped into a successful stroke segment set, completing the first mapping from the successful stroke segment set to the stroke set: each stroke segment is mapped to a successfully matched stroke order; the unmatched stroke segments are grouped into a failed stroke segment set, and the following second stroke order matching operation is performed on the failed stroke segment set.

[0160] The second stroke order matching steps are: for each stroke segment in the failed stroke segment set, respectively calculate the distance between the centroid of the stroke segment and the centroid of each stroke in the stroke set, and form a distance set; find the minimum distance in the distance set; find the stroke order of the stroke corresponding to the minimum distance in the stroke set, and determine that the stroke order matches the stroke segment successfully; complete the second mapping from the failed stroke segment set to the stroke set: map each stroke segment to the stroke order that matches successfully.

[0161] The embodiment of the present invention performs a first stroke order matching and a second stroke order matching on handwritten Chinese characters, thereby ensuring that each stroke segment in the handwritten Chinese characters has a matching stroke order.

[0162] The stroke segment fusion step is: merging the first mapping and the second mapping into a third mapping, and forming a stroke segment set to be fused of the stroke order with two or more stroke segments mapped to the same stroke order in the third mapping; performing a stroke segment fusion operation on each stroke segment set to be fused respectively: fusion of the stroke segments in each stroke segment set to be fused into one stroke segment.

[0163] like Figure 3 As shown, the segment fusion operation includes a segment encoding step, a segment flipping step, a segment fusion step, and a fusion completion step:

[0164] Segment encoding step: Taking the starting point as the starting point, traverse each segment in the fused segment set, and calculate the direction code of each handwriting pixel except the starting point according to the eight-neighborhood direction coding rule. The various direction codes constitute the eight-neighborhood coding chain of the segment in the traversal order of the traversal operation.

[0165] Flipping segment step: for each segment in the fused segment set, search the image of the segment in the third mapping, and search the legal direction code of the corresponding stroke in the stroke set according to the image of the segment; determine whether the eight-neighborhood code chain of the segment is included in the legal direction code, if not, flip the segment, put the flipped segment into the segment set to be fused, and delete the segment before flipping; otherwise, do not flip the segment, and keep it in the segment set to be fused.

[0166] Method for flipping a pen segment: If a pen segment u contains n pen pixels, that is, u: {p0, p1, ..., p n-2 , p n-1}, then the result of flipping the segment u is u′: {p n-1 , p n-2 , ..., p1, p0};

[0167] Segment fusion step: randomly select two segments in the segment set to be fused: the first segment and the second segment, and calculate the distances D1, D2, D3 and D4 for the starting pixel point P1 and the ending pixel point P2 of the first segment, and the starting pixel point P3 and the ending pixel point P4 of the second segment; D1 is the distance between P1 and P3, D2 is the distance between P1 and P4, D3 is the distance between P2 and P3, and D4 is the distance between P2 and P4; if the smallest distance among D1-D4 is less than the set threshold, the two points with the closest distance among P1-P4 are taken for fusion, and the fused segment is put into the segment set to be fused; otherwise, it is determined that the fusion fails, and the segment that fails to be fused is deleted; the segment that fails to be fused is the segment in the first segment and the second segment that belongs to the failed segment set.

[0168] Fusion completion step: Repeat step E2 for the segment set to be fused until it is fused into one segment.

[0169] Compared with the prior art that realizes stroke segment fusion by setting thresholds, the embodiment of the present invention fully considers the different stroke types of Chinese characters and realizes stroke segment fusion through the eight-neighborhood coding chains of different strokes and the legal direction coding of different strokes. The stroke segment fusion method of the embodiment of the present invention is more detailed and comprehensive, and can achieve the beneficial effect of high stroke segment fusion accuracy.

[0170] The restoration and normalization step is: performing a restoration and normalization operation on the segmented image after the stroke segment fusion operation is performed to obtain an annotated Chinese character image with stroke order annotation completed, including the following sub-steps:

[0171] F0: solve the inverse matrix of the perspective projection matrix in step B0;

[0172] F1: Use the inverse matrix to restore and normalize the pixels in each stroke segment; the image size after restoration and normalization is still 500*500 pixels, and all stroke segments are restored to their positions before restoration and normalization;

[0173] F2: Mark the stroke order of all strokes, such as Figure 5 As shown, numbers 1 to 6 are the stroke order of the stroke segment.

[0174] The stroke order recognition system for handwritten Chinese character images provided by the embodiment of the present invention comprises a stroke set acquisition module, a preprocessing module, a first stroke order matching module, a second stroke order matching module, a stroke segment fusion module and a restoration and normalization module.

[0175] The stroke set acquisition module is used to establish a standardized Chinese character library for all Chinese characters in the national standard first-level character library according to the eight-neighborhood direction encoding rules. The standardized Chinese character library is composed of a standard stroke set for each Chinese character. The standard stroke set includes the stroke order, centroid, illegal direction encoding and legal direction encoding of each stroke; the standard stroke set of handwritten Chinese characters in the original handwritten Chinese character image is searched in the standardized Chinese character library, hereinafter referred to as the stroke set.

[0176] The rules for illegal direction encoding are:

[0177] If the main direction of the stroke is coded as 0, the illegal directions are coded as 2 and 6;

[0178] If the main direction of the stroke is coded as 1, the illegal directions are coded as 3 and 7;

[0179] If the main direction of the stroke is coded as 2, the illegal directions are coded as 0 and 4;

[0180] If the main direction of the stroke is coded as 3, the illegal directions are coded as 1 and 5;

[0181] If the main direction of the stroke is coded as 4, the illegal directions are coded as 2 and 6;

[0182] If the main direction of the stroke is coded as 5, the illegal directions are coded as 3 and 7;

[0183] If the main direction of the stroke is coded as 6, the illegal directions are coded as 0 and 4;

[0184] If the main direction of the stroke is coded as 7, the illegal directions are coded as 1 and 5;

[0185] The rules for legal direction encoding are:

[0186] If the main direction of the stroke is coded as 0, then the legal direction codes are 0, 1, and 7;

[0187] If the main direction of the stroke is coded as 1, then the legal directions are coded as 0, 1, and 2;

[0188] If the main direction of the stroke is coded as 2, then the legal directions are coded as 1, 2, and 3;

[0189] If the main direction of the stroke is coded as 3, then the legal directions are coded as 2, 3, and 4;

[0190] If the main direction of the stroke is coded as 4, then the legal directions are coded as 3, 4, and 5;

[0191] If the main direction of the stroke is coded as 5, then the legal directions are coded as 4, 5, and 6;

[0192] If the main direction of the stroke is coded as 6, then the legal directions are coded as 5, 6, and 7;

[0193] If the main direction of the stroke is coded as 7, then the legal direction codes are 0, 6, and 7;

[0194] The main direction code of a stroke is the direction code that appears most frequently in the eight-neighborhood code chain of the stroke.

[0195] The preprocessing module is used to perform Chinese character skeleton extraction operation on the original handwritten Chinese character image to obtain a Chinese character skeleton image; and to perform segmentation operation on the Chinese character skeleton to obtain a segmented image containing multiple stroke segments.

[0196] The Chinese character skeleton extraction operation is as follows: performing a normalization operation on the original handwritten Chinese character image; performing a refinement operation on the normalized Chinese character image to obtain a Chinese character skeleton image.

[0197] Among them, the method for obtaining the original handwritten Chinese character image is: designing squares of uniform size on A4 size paper; asking the writer to write a Chinese character in each square; using a camera to photograph the written paper to obtain the original image; performing contour extraction on the original image, segmenting each square, and extracting the handwritten Chinese characters in each square; the handwritten Chinese character image in a single square is the original handwritten Chinese character image.

[0198] The normalization operation is as follows: perform contour recognition on a single Chinese character image to extract the enclosing rectangle of the Chinese character; expand and extract a square enclosing box from the enclosing rectangle; use the perspective projection matrix to segment and extract the square enclosing box, and scale it to a size of 500*500 pixels.

[0199] The thinning operation is as follows: grayscale and binarization operations are performed on the Chinese character image in turn; the Zhang-Suen thinning algorithm is used to process the Chinese character image to obtain a preliminary thinned Chinese character skeleton; the index table thinning algorithm is used to process the preliminary thinned Chinese character skeleton again to obtain the final Chinese character skeleton image. Any handwriting pixel point of the Chinese character skeleton image has no more than 4 adjacent handwriting pixels in its eight-neighborhood.

[0200] The split operation is:

[0201] (1) deleting intersection points in the Chinese character skeleton image, storing the intersection points in the first list L1, deleting redundant noise points, and segmenting the Chinese character strokes into multiple stroke segments to obtain a preliminary segmented image; wherein the handwriting pixel points whose number of handwriting pixel points contained in their eight neighborhoods is greater than or equal to 3 are intersection points;

[0202] (2) Traverse the preliminary segmented image, find the starting and ending points of all the strokes, and store them in the second list L2;

[0203] (3) Take out the starting and ending points of each segment from the second list L2, traverse the segment, find the inflection point in the segment, and delete the inflection point;

[0204] (4) Traverse the first list L1 of intersection points, connect two intersection points whose distance is less than the threshold, and if the connected segment has no inflection point, retain the connected segment; if there is an inflection point, cancel the connection; the threshold can be set to 6.

[0205] After the segmentation operation is completed, the first starting and ending points L and the second starting and ending points M of the pen segment have only one adjacent handwriting pixel point in their eight neighborhoods, and the other handwriting pixels have only two adjacent handwriting pixels in their eight neighborhoods.

[0206] The first stroke order matching module is used to perform stroke order matching operations on the stroke segments in the segmented image according to the stroke set through the branch and bound method, and the successfully matched stroke segments are combined into a successful stroke segment set to complete the first mapping from the successful stroke segment set to the stroke set: each stroke segment is mapped to a successfully matched stroke order; the unmatched stroke segments are combined into a failed stroke segment set, and the operations in the second stroke order matching module described below are performed on the failed stroke segment set.

[0207] The first stroke order matching module includes the following submodules:

[0208] Matching weight calculation submodule: used to calculate the main direction code of each stroke segment, and compare the illegal direction codes of all strokes in the stroke set respectively: if the main direction code of the stroke segment does not belong to the illegal direction code of one of the strokes, then calculate the matching weight of the stroke segment and the stroke, and continue to compare the next stroke; otherwise, determine that the stroke segment does not match the stroke, and continue to compare the next stroke; until all strokes are compared;

[0209] The main direction code of a segment is the direction code that appears most frequently in the eight-neighborhood code chain of the segment.

[0210] The matching weight is the distance between the centroid of the segment and the centroid of the stroke.

[0211] Matching stroke order submodule: used to match the stroke order by the branch and bound method according to all the matching weights of each stroke segment, group the successfully matched stroke segments into a successful stroke segment set, and complete the first mapping from the successful stroke segment set to the stroke set: map each stroke segment to the successfully matched stroke order; group the unmatched stroke segments into a failed stroke segment set, and perform the following operations in the second stroke order module on the failed stroke segment set.

[0212] The second stroke order matching module is used to calculate the distance between the centroid of each stroke in the stroke set and the centroid of each stroke in the stroke set for each stroke segment in the failed stroke segment set, and form a distance set; find the minimum distance in the distance set; find the stroke order of the stroke corresponding to the minimum distance in the stroke set, and determine that the stroke order matches the stroke segment successfully; complete the second mapping from the failed stroke segment set to the stroke set: map each stroke segment to the stroke order that matches successfully.

[0213] The stroke segment fusion module is used to merge the first mapping and the second mapping into a third mapping, and to form a stroke segment set to be merged of the stroke order with more than two stroke segments mapped to the same stroke order in the third mapping; and to perform a stroke segment fusion operation on each stroke segment set to be merged respectively: the stroke segments in each stroke segment set to be merged are merged into one stroke segment.

[0214] The stroke fusion module includes the following submodules:

[0215] Segment coding submodule: It is used to perform traversal operations on each segment in the fused segment set starting from the starting point, and calculate the direction coding of each handwriting pixel point except the starting point according to the eight-neighborhood direction coding rule. The various direction codes constitute the eight-neighborhood coding chain of the segment in the traversal order of the traversal operation.

[0216] The flipping segment submodule is used to treat each segment in the fused segment set, search for the image of the segment in the third mapping, and search for the legal direction code of the corresponding stroke in the stroke set according to the image of the segment; determine whether the eight-neighborhood code chain of the segment is included in the legal direction code. If not, flip the segment, put the flipped segment into the set of segments to be fused, and delete the segment before flipping; otherwise, do not flip the segment, and keep it in the set of segments to be fused.

[0217] Method for flipping a pen segment: If a pen segment u contains n pen pixels, that is, u: {p0, p1, ..., p n-2 , p n-1}, then the result of flipping the segment u is u′: {p n-1 , p n-2 , ..., p1, p0};

[0218] Segment fusion submodule: used to randomly select two segments in the segment set to be fused: the first segment and the second segment, and calculate the distances D1, D2, D3 and D4 for the starting pixel point P1 and the ending pixel point P2 of the first segment, and the starting pixel point P3 and the ending pixel point P4 of the second segment; D1 is the distance between P1 and P3, D2 is the distance between P1 and P4, D3 is the distance between P2 and P3, and D4 is the distance between P2 and P4; if the smallest distance among D1 to D4 is less than the set threshold, the two points with the closest distance among P1 to P4 are taken for fusion, and the fused segment is put into the segment set to be fused; otherwise, it is determined that the fusion fails, and the segment that fails to be fused is deleted; the segment that fails to be fused is the segment in the first segment and the second segment that belongs to the failed segment set.

[0219] Fusion completion submodule: used to repeatedly execute the operations in the fusion segment submodule on the fused segment set until it is fused into one segment.

[0220] The restoration and normalization module is used to perform a restoration and normalization operation on the segmented image after the stroke segment fusion operation is performed, so as to obtain an annotated Chinese character image with the stroke order annotated.

[0221] The normalization operation is restored as:

[0222] (1) Solving the inverse matrix of the perspective projection matrix in the normalization operation in the preprocessing module;

[0223] (2) Use the inverse matrix to restore and normalize the pixels in each stroke segment; the image size after restoration and normalization is still 500*500 pixels, and all stroke segments are restored to their positions before restoration and normalization;

[0224] (3) Mark the stroke order of all strokes.

[0225] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for stroke order recognition of handwritten Chinese character images, characterized in that: The following steps are involved: (1) Preprocessing step: performing a Chinese character skeleton extraction operation on the original handwritten Chinese character image to obtain a Chinese character skeleton image; performing a segmentation operation on the Chinese character skeleton to obtain a segmented image containing multiple stroke segments; the original handwritten Chinese character image contains only one handwritten Chinese character; (2) A first stroke order matching step: performing a stroke order matching operation on the stroke segments in the segmented image according to the stroke set by using the branch and bound method, forming a successful stroke segment set with the successfully matched stroke segments, and completing a first mapping from the successful stroke segment set to the stroke set: mapping each stroke segment to a successfully matched stroke order; forming a failed stroke segment set with the failed stroke segments, and performing the following second stroke order matching step on the failed stroke segment set; The method for obtaining the stroke set is: searching in a standard Chinese character library to obtain a standard stroke set of the handwritten Chinese characters in the original handwritten Chinese character image, referred to as the stroke set; The method for establishing the standardized Chinese character library is as follows: establishing a standardized Chinese character library for all Chinese characters in the national standard first-level character library according to the eight-neighborhood direction encoding rule, wherein the standardized Chinese character library is composed of a standard stroke set of each Chinese character, and the standard stroke set includes the stroke order and centroid of each stroke; (3) A second stroke order matching step: for each stroke segment in the failed stroke segment set, respectively calculating the distance between the centroid of the stroke segment and the centroid of each stroke in the stroke set, and forming a distance set; finding the minimum distance in the distance set; finding the stroke order of the stroke corresponding to the minimum distance in the stroke set, and determining whether the stroke order matches the stroke segment successfully; Complete the second mapping from the failed stroke set to the stroke set: map each stroke segment to a stroke order that matches successfully; (4) a stroke segment fusion step: merging the first mapping and the second mapping into a third mapping, and forming a stroke segment set to be fused of the stroke order with two or more stroke segments mapped to the same stroke order in the third mapping; performing a stroke segment fusion operation on each of the stroke segment sets to be fused respectively: fusion of the stroke segments in each of the stroke segment sets to be fused into one stroke segment; Any handwriting pixel point of the Chinese character skeleton image in the preprocessing step has no more than 4 adjacent handwriting pixel points in its eight neighborhoods; the starting and ending points of the stroke segment in the preprocessing step have no more than 1 adjacent handwriting pixel point in its eight neighborhoods, and other handwriting pixel points have no more than 2 adjacent handwriting pixel points in their eight neighborhoods; The standard stroke set in the standard Chinese character library also includes illegal direction codes and legal direction codes for each stroke; The rules for illegal direction encoding are: If the main direction of the stroke is coded as 0, the illegal directions are coded as 2 and 6; If the main direction of the stroke is coded as 1, the illegal directions are coded as 3 and 7; If the main direction of the stroke is coded as 2, the illegal directions are coded as 0 and 4; If the main direction of the stroke is coded as 3, the illegal directions are coded as 1 and 5; If the main direction of the stroke is coded as 4, the illegal directions are coded as 2 and 6; If the main direction of the stroke is coded as 5, the illegal directions are coded as 3 and 7; If the main direction of the stroke is coded as 6, the illegal directions are coded as 0 and 4; If the main direction of the stroke is coded as 7, the illegal directions are coded as 1 and 5; The rules for legal direction encoding are: If the main direction of the stroke is coded as 0, then the legal direction codes are 0, 1, and 7; If the main direction of the stroke is coded as 1, then the legal directions are coded as 0, 1, and 2; If the main direction of the stroke is coded as 2, then the legal directions are coded as 1, 2, and 3; If the main direction of the stroke is coded as 3, then the legal directions are coded as 2, 3, and 4; If the main direction of the stroke is coded as 4, then the legal directions are coded as 3, 4, and 5; If the main direction of the stroke is coded as 5, then the legal directions are coded as 4, 5, and 6; If the main direction of the stroke is coded as 6, then the legal directions are coded as 5, 6, and 7; If the main direction of the stroke is coded as 7, then the legal direction codes are 0, 6, and 7; The main direction code of the stroke is the direction code that appears most frequently in the eight-neighborhood code chain of the stroke.

2. The stroke order recognition method for handwritten Chinese character images according to claim 1, characterized in that: The stroke segment matching operation in the first stroke order matching step comprises the following steps: (1) Matching weight calculation step: Calculate the main direction code of each of the stroke segments, and compare the illegal direction codes of all the strokes in the stroke set respectively: If the main direction code of the stroke segment does not belong to the illegal direction code of one of the strokes, calculate the matching weight of the stroke segment and the stroke, and continue to compare with the next stroke; otherwise, determine that the stroke segment does not match the stroke, and continue to compare with the next stroke; until all strokes are compared; The main direction code of the segment is the direction code that appears most frequently in the eight-neighborhood code chain of the segment; The matching weight is the distance between the centroid of the stroke segment and the centroid of the stroke; (2) Stroke order matching step: According to all matching weights of each stroke segment, the first mapping is completed by using the branch and bound method.

3. The stroke order recognition method for handwritten Chinese character images according to claim 1, characterized in that: The segment fusion operation in the segment fusion step includes the following steps: (1) Segment encoding step: starting from the starting point, traversing each segment in the set of stroke segments to be merged, calculating the direction code of each stroke pixel except the starting point according to the eight-neighborhood direction coding rule, and each direction code constitutes an eight-neighborhood coding chain of the stroke segment in the traversal order of the traversal operation; (2) Flipping a segment step: for each segment in the set of segments to be merged, search for the image of the segment in the third mapping, and search for the legal direction code of the corresponding stroke in the stroke set according to the image of the segment; determine whether the eight-neighborhood code chain of the segment is included in the legal direction code; if not, flip the segment, put the flipped segment into the set of segments to be merged, and delete the segment before flipping; otherwise, do not flip the segment, and keep it in the set of segments to be merged; (3) Segment fusion step: randomly select two segments from the set of segments to be fused: the first segment and the second segment, and calculate the distances D1, D2, D3 and D4 for the starting pixel point P1 and the ending pixel point P2 of the first segment, and the starting pixel point P3 and the ending pixel point P4 of the second segment; D1 is the distance between P1 and P3, D2 is the distance between P1 and P4, D3 is the distance between P2 and P3, and D4 is the distance between P2 and P4; if the smallest distance among D1 to D4 is less than a set threshold, then select the two points with the closest distance among P1 to P4 for fusion, and put the fused segment into the set of segments to be fused; otherwise, it is determined that the fusion fails, and the segment that fails to be fused is deleted; the segment that fails to be fused is the segment of the first segment and the second segment that belongs to the set of failed segments; (4) Fusion completion step: Repeat the fusion step for the segment set to be fused until it is fused into one segment.

4. A stroke order recognition system for handwritten Chinese character images, characterized in that: include: A stroke set acquisition module is used to establish a standardized Chinese character library for all Chinese characters in the national standard first-level character library according to the eight-neighborhood direction encoding rule, wherein the standardized Chinese character library is composed of a standard stroke set of each Chinese character, and the standard stroke set includes the stroke order and centroid of each stroke; and to search the standard stroke set of the handwritten Chinese characters in the original handwritten Chinese character image in the standardized Chinese character library, hereinafter referred to as the stroke set; The original handwritten Chinese character image contains only one handwritten Chinese character; Preprocessing module: used for performing Chinese character skeleton extraction operation on the original handwritten Chinese character image to obtain a Chinese character skeleton image; performing segmentation operation on the Chinese character skeleton to obtain a segmented image containing multiple stroke segments; The first stroke order matching module is used to perform stroke order matching operations on the stroke segments in the segmented image according to the stroke set by using the branch and bound method, and the successfully matched stroke segments are grouped into a successful stroke segment set, and the first mapping from the successful stroke segment set to the stroke set is completed: each stroke segment is mapped to a successfully matched stroke order; the unmatched stroke segments are grouped into a failed stroke segment set, and the operations in the second stroke order matching module are performed on the failed stroke segment set; The second stroke order matching module is used to calculate the distance between the centroid of each stroke in the stroke set and the centroid of each stroke in the stroke set for each stroke in the failed stroke set, and form a distance set; find the minimum distance in the distance set; find the stroke order of the stroke corresponding to the minimum distance in the stroke set, and determine whether the stroke order matches the stroke successfully; Complete the second mapping from the failed stroke set to the stroke set: map each stroke segment to a stroke order that matches successfully; The stroke segment fusion module is used to merge the first mapping and the second mapping into a third mapping, and to form a stroke segment set to be merged of the stroke sequence with two or more stroke segments mapped to the same stroke sequence in the third mapping; and to perform a stroke segment fusion operation on each of the stroke segment sets to be merged: to merge the stroke segments in each of the stroke segment sets to be merged into one stroke segment; Any handwriting pixel point of the Chinese character skeleton image in the preprocessing module has no more than 4 adjacent handwriting pixel points in its eight neighborhoods; the starting and ending points of the stroke segments in the preprocessing module have no more than 1 adjacent handwriting pixel point in its eight neighborhoods, and other handwriting pixel points have no more than 2 adjacent handwriting pixel points in their eight neighborhoods; The standard stroke set in the stroke set acquisition module also includes an illegal direction code and a legal direction code of each stroke; The rules for illegal direction encoding are: If the main direction of the stroke is coded as 0, the illegal directions are coded as 2 and 6; If the main direction of the stroke is coded as 1, the illegal directions are coded as 3 and 7; If the main direction of the stroke is coded as 2, the illegal directions are coded as 0 and 4; If the main direction of the stroke is coded as 3, the illegal directions are coded as 1 and 5; If the main direction of the stroke is coded as 4, the illegal directions are coded as 2 and 6; If the main direction of the stroke is coded as 5, the illegal directions are coded as 3 and 7; If the main direction of the stroke is coded as 6, the illegal directions are coded as 0 and 4; If the main direction of the stroke is coded as 7, the illegal directions are coded as 1 and 5; The rules for legal direction encoding are: If the main direction of the stroke is coded as 0, then the legal direction codes are 0, 1, and 7; If the main direction of the stroke is coded as 1, then the legal directions are coded as 0, 1, and 2; If the main direction of the stroke is coded as 2, then the legal directions are coded as 1, 2, and 3; If the main direction of the stroke is coded as 3, then the legal directions are coded as 2, 3, and 4; If the main direction of the stroke is coded as 4, then the legal directions are coded as 3, 4, and 5; If the main direction of the stroke is coded as 5, then the legal directions are coded as 4, 5, and 6; If the main direction of the stroke is coded as 6, then the legal directions are coded as 5, 6, and 7; If the main direction of the stroke is coded as 7, then the legal direction codes are 0, 6, and 7; The main direction code of the stroke is the direction code that appears most frequently in the eight-neighborhood code chain of the stroke.

5. The stroke order recognition system for handwritten Chinese character images according to claim 4, characterized in that: The first stroke order matching module includes the following submodules: A matching weight calculation submodule is used to calculate the main direction code of each of the stroke segments, and compare the illegal direction codes of all the strokes in the stroke set respectively: if the main direction code of the stroke segment does not belong to the illegal direction code of one of the strokes, then the matching weight of the stroke segment and the stroke is calculated, and the comparison is continued with the next stroke; otherwise, it is determined that the stroke segment does not match the stroke, and the comparison is continued with the next stroke; until the comparison of all strokes is completed; The main direction code of the segment is the direction code that appears most frequently in the eight-neighborhood code chain of the segment; The matching weight is the distance between the centroid of the stroke segment and the centroid of the stroke; Matching stroke order submodule: used to complete the first mapping through the branch and bound method according to all matching weights of each stroke segment.

6. The stroke order recognition system for handwritten Chinese character images according to claim 4, characterized in that: The stroke fusion module Includes the following submodules: The stroke segment coding submodule is used to perform a traversal operation on each stroke segment in the stroke segment set to be fused, starting from the starting point, and calculating the direction code of each stroke pixel point except the starting point according to the eight-neighborhood direction coding rule. Each of the direction codes constitutes an eight-neighborhood coding chain of the stroke segment in the traversal order of the traversal operation; The flipping segment submodule is used to search the image of each segment in the set of segments to be merged in the third mapping, and search the legal direction code of the corresponding stroke in the stroke set according to the image of the segment; determine whether the eight-neighborhood code chain of the segment is included in the legal direction code, and if not, flip the segment, put the flipped segment into the set of segments to be merged, and delete the segment before flipping; otherwise, do not flip the segment, and keep it in the set of segments to be merged; The fusion segment submodule is used to randomly select two segments from the set of segments to be fused: the first segment and the second segment, and calculate the distances D1, D2, D3 and D4 for the starting pixel point P1 and the ending pixel point P2 of the first segment, and the starting pixel point P3 and the ending pixel point P4 of the second segment; D1 is the distance between P1 and P3, D2 is the distance between P1 and P4, D3 is the distance between P2 and P3, and D4 is the distance between P2 and P4; if the smallest distance among D1 to D4 is less than the set threshold, the two points with the closest distance among P1 to P4 are taken for fusion, and the fused segment is put into the set of segments to be fused; otherwise, it is determined that the fusion fails, and the segment that fails to be fused is deleted; the segment that fails to be fused is the segment in the first segment and the second segment that belongs to the set of failed segments; Fusion completion submodule: used to repeatedly execute the operations in the fusion segment submodule on the set of segments to be fused until they are fused into one segment.

7. A stroke order recognition device for handwritten Chinese character images, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the stroke order recognition method for handwritten Chinese character images as described in any one of claims 1 to 3 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for stroke order recognition of a handwritten Chinese character image as described in any one of claims 1 to 3 is implemented.

Citation Information

Cited By

  • Method, system and equipment for identifying handwritten characters with different writing methods on line and storage medium

    CN117789309A