Writing assistance system and method based on the grid and computer vision

Abstract

The present invention discloses a writing assistance system and method based on a grid and computer vision, belonging to the field of image recognition technology, and specifically comprises: collecting written text images uploaded by users, dividing a writing paper area and a background area; performing direction correction on the written text image according to the placement direction of the writing paper; selecting a text area in any grid in the writing paper, detecting the distance between the upper and lower parts or the left and right parts of the text, and if the distance is greater than a preset distance, judging that the text structure is not standardized; being used to select a text area in any row, horizontally enlarging the text area in each grid with the vertical line where the center point of the grid is located as the starting point, until the text areas in adjacent grids are in contact, generating a continuous text pixel strip area, calculating the slope of the strip area fitting straight line, and judging that the text handwriting of the row is insufficient in continuity; the present invention improves the efficiency of evaluating written handwriting.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a writing assistance system and method based on a grid and computer vision. Background Art

[0002] In traditional calligraphy teaching, the evaluation of students' handwriting quality mainly relies on the subjective judgment and personal experience of teachers. This method is not only inefficient, but also difficult to achieve accurate evaluation of each student's handwriting details. Since each teacher's evaluation criteria and aesthetic concepts are different, there may be significant differences in the evaluation of students' handwriting quality between different teachers, which makes the teaching process lack uniformity and objectivity. In addition, traditional methods usually require teachers to spend a lot of time and energy on correction and guidance, which not only increases the workload of teachers, but also limits the number and scope of students they can pay attention to.

[0003] With the development of computer technology, it has become possible to automatically analyze and evaluate written text using computer vision technology. Computer vision technology can convert written text into digital information through steps such as image acquisition, preprocessing, and feature extraction, thereby realizing automatic evaluation of writing quality. This method not only improves the efficiency of evaluation, but also can reduce the influence of human factors to a certain extent, making the evaluation results more objective and fair.

[0004] However, most existing writing assistance systems focus on the recognition and analysis of the overall character shape, and there are still certain limitations in the micro-level evaluation of the coherence and structural norms of the handwriting during the writing process. Therefore, how to design a system that can comprehensively and meticulously evaluate the quality of students' handwriting has become an urgent problem to be solved in the current field of calligraphy teaching. Summary of the invention

[0005] The purpose of the present invention is to provide a writing assistance system and method based on the rice character grid and computer vision to solve the following technical problems:

[0006] Most existing writing assistance systems focus on the recognition and analysis of the overall character shape, and there are still certain limitations in the micro-level evaluation of handwriting continuity, structural standardization, etc. during the writing process.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] The writing assistance system based on the Mizi Zhengti grid and computer vision includes:

[0009] An image acquisition module is used to acquire written text images uploaded by users, and pre-process the written text images to divide the written text images into a writing paper area and a background area;

[0010] An image processing module, used for identifying the edge of the writing paper area and performing direction correction on the writing text image according to the placement direction of the writing paper;

[0011] A single handwriting evaluation module is used to select a text area in any grid on the writing paper, collect the starting point and ending point of each stroke based on the dot matrix intelligent pen, mark them as pending points, obtain a number of pending points farthest from the center point of the grid, define them as feature points, connect adjacent feature points in sequence to generate a feature graph, obtain the area of ​​the feature graph exceeding the preset positive grid area, and when the area is greater than a preset threshold, identify whether it is a left-right structure or an upper-lower structure. If not, it is judged that the writing of the text occupies a large grid; if so, detect the distance between the upper and lower parts or the left and right parts of the text. If the distance is greater than the preset distance, it is judged that the text structure is not standardized;

[0012] The coherent handwriting evaluation module is used to select the text area of ​​any line, horizontally enlarge the text area in each grid with the vertical line where the center point of the grid is located as the starting point, until the text areas in adjacent grids touch each other, generate a continuous text pixel strip area, calculate the slope of the fitting straight line of the strip area, and when the difference between the slope and 0 is greater than a preset threshold, it is judged that the text handwriting of the line is insufficient in coherence.

[0013] As a further solution of the present invention: the pretreatment process is:

[0014] The written text image is grayscaled to generate a grayscale image, the contrast of the grayscale image is increased, and the grayscale image is converted into a binary image by setting a grayscale threshold. The binary image is denoised based on Gaussian filtering, and edge detection is performed on the denoised binary image to segment the writing paper area from the background area.

[0015] As a further solution of the present invention: the preset positive ladder grid area is composed of 6 calibration points, 2 of which are set in the upper half of the grid, and the other 4 points are set in the lower half of the grid. The two points in the upper half are positioned at the midpoints of the oblique lines of the left and right upper half grids; two of the points in the lower half are positioned at the intersection of the inscribed circle of the square grid and the oblique line of the grid, that is, the outer points, and the remaining two points in the lower half are positioned at the two golden section points of the line connecting the two outer points.

[0016] As a further solution of the present invention: the process of correcting the direction of the written text image is:

[0017] The edge lines of the writing paper area are obtained, and the slope differences between the four edge lines of the writing paper and the boundary lines of the writing text image are calculated respectively. The writing text image is offset corrected according to the average value of the slope differences.

[0018] As a further solution of the present invention: the process of detecting the distance between the upper and lower parts or the left and right parts of the text is:

[0019] Traverse the text area according to a set direction, the set direction is a vertical direction or a horizontal direction, count the grayscale value of each column of pixels detected, the grayscale value of the text pixel is 0, the grayscale value of the pixel in the writing paper area is 255, select the first pixel column whose number of text pixels accounts for a ratio greater than a preset ratio k of the total number of pixels in the column, mark it as the start column of the text, continue to traverse the text area, when a pixel column whose number of text pixels accounts for a ratio less than k is detected, mark it as a pending column, obtain the pixel distance D1 between the pending column and the start column, if the pixel distance D1 is greater than or equal to the preset distance d1, and continue to detect along the set direction and no text pixels are found, mark the pending column as the end column, and the text is a monomer structure;

[0020] If the pixel distance is less than d, continue to traverse the text area. When a pixel column with a text pixel ratio greater than k is detected again, it is marked as the second starting column. The distance D2 between the second starting column and the pending column is obtained. If the distance D2 is greater than the preset distance d2, it is determined that the structure of the text is not standardized.

[0021] As a further solution of the present invention: the specific process of judging whether the handwriting continuity of a text is insufficient is as follows:

[0022] For any selected grid, select the vertical line where the center point of the grid is located, obtain the distance of all pixels in the text relative to the vertical line, multiply the distance of each pixel point relative to the vertical line by the same ratio, and continue to increase the value of the ratio until adjacent grids are in contact with each other, to obtain a strip area of ​​text pixels, obtain the corresponding fitting straight line equation through median filtering of the strip area, and obtain the slope of the straight line equation.

[0023] As a further solution of the present invention: the process of generating the characteristic graph is:

[0024] First, select the three pending points that are farthest from the center point of the grid, connect the adjacent pending points to generate a pending feature graph, and then select the points with the farthest distance from the remaining pending points one by one to add to the lines of the feature graph, until the angle corresponding to the original feature point in the line formed by the newly added feature point becomes smaller, then the added point is extracted, and the line formed by the original feature point is the feature graph of the text.

[0025] The writing assistance method based on the cross-shaped grid and computer vision is applied to the above-mentioned writing assistance method based on the cross-shaped grid and computer vision, comprising the following steps:

[0026] Collecting written text images uploaded by users, and preprocessing the written text images to divide the written text images into a writing paper area and a background area;

[0027] Identify the edge of the writing paper area, and perform direction correction on the written text image according to the placement direction of the writing paper;

[0028] Select a text area in any grid on the writing paper, collect the starting point and ending point of each stroke based on the dot matrix smart pen, mark them as pending points, obtain several pending points farthest from the center point of the grid, define them as feature points, connect adjacent feature points in sequence to generate feature graphs, obtain the area of ​​the feature graph exceeding the preset positive grid area, and when the area is greater than a preset threshold, identify whether it is a left-right structure or an upper-lower structure. If not, it is judged that the writing of the text occupies a large grid; if so, detect the distance between the upper and lower parts or the left and right parts of the text. If the distance is greater than the preset distance, it is judged that the text structure is not standardized;

[0029] Select the text area of ​​any row, and horizontally enlarge the text area in each grid with the vertical line where the center point of the grid is located as the starting point until the text areas in adjacent grids touch each other, generate a continuous text pixel strip area, calculate the slope of the fitting line of the strip area, and when the difference between the slope and 0 is greater than the preset threshold, it is judged that the text handwriting of the row lacks continuity.

[0030] Beneficial effects of the present invention:

[0031] The present invention uses computer vision technology to automatically analyze and evaluate written text, which significantly improves the efficiency of evaluation. Compared with traditional manual evaluation methods, the system can process a large number of written text images in a short time. The system not only focuses on the recognition and analysis of the overall character shape, but also goes deep into the micro-level such as the continuity of handwriting and structural standardization during the writing process. By accurately collecting and analyzing the landing point and the end point of each stroke, the system can generate a feature graph and calculate its area exceeding the preset positive grid area, so as to judge whether the writing grid of the text is large and whether the structure is standardized. This comprehensive and detailed evaluation method helps students to understand their own writing problems more clearly and make targeted improvements. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention will be further described below in conjunction with the accompanying drawings.

[0033] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1 As shown, the present invention is a writing assistance system and method based on the rice character regular ladder grid and computer vision:

[0036] An image acquisition module is used to acquire written text images uploaded by users, and pre-process the written text images to divide the written text images into a writing paper area and a background area;

[0037] An image processing module, used for identifying the edge of the writing paper area and performing direction correction on the writing text image according to the placement direction of the writing paper;

[0038] A single handwriting evaluation module is used to select a text area in any grid on the writing paper, collect the starting point and ending point of each stroke based on the dot matrix intelligent pen, mark them as pending points, obtain a number of pending points farthest from the center point of the grid, define them as feature points, connect adjacent feature points in sequence to generate a feature graph, obtain the area of ​​the feature graph exceeding the preset positive grid area, and when the area is greater than a preset threshold, identify whether it is a left-right structure or an upper-lower structure. If not, it is judged that the writing of the text occupies a large grid; if so, detect the distance between the upper and lower parts or the left and right parts of the text. If the distance is greater than the preset distance, it is judged that the text structure is not standardized;

[0039] The coherent handwriting evaluation module is used to select the text area of ​​any line, horizontally enlarge the text area in each grid with the vertical line where the center point of the grid is located as the starting point, until the text areas in adjacent grids touch each other, generate a continuous text pixel strip area, calculate the slope of the fitting straight line of the strip area, and when the difference between the slope and 0 is greater than a preset threshold, it is judged that the text handwriting of the line is insufficient in coherence.

[0040] The present invention can upload the completed work to the system in the following ways:

[0041] Take photos and upload: Use mobile devices to take pictures of your work and upload them directly to the system.

[0042] Pen upload: Students can directly upload their writing content through the upload tool. This method is particularly suitable for calligraphy practice and calligraphy homework, which can capture the handwriting and pressure changes of writing, and provide detailed data for subsequent evaluation and analysis;

[0043] Teachers can view students' handwriting pictures and teachers' personalized comments to understand students' writing skills and styles. The AI ​​review report and learning competency analysis provided by the system help teachers comprehensively evaluate students' learning performance and room for improvement.

[0044] In another preferred embodiment of the present invention, the pretreatment process is:

[0045] The written text image is grayscaled to generate a grayscale image, the contrast of the grayscale image is increased, and the grayscale image is converted into a binary image by setting a grayscale threshold. The binary image is denoised based on Gaussian filtering, and edge detection is performed on the denoised binary image to segment the writing paper area from the background area.

[0046] In another preferred embodiment of the present invention, the preset regular grid area is composed of 6 calibration points, and the grid area is divided by a cross, providing an accurate reference frame. Practitioners can ensure that the upper, lower, left and right parts of the font are coordinated with each other by observing these lines to achieve an overall balanced effect.

[0047] The two points above are precisely located at the midpoints of the upper left and right half-grid diagonal lines, providing a reference for starting and positioning. Pulling out the strokes from the points solves the problems of difficulty in starting and difficulty in controlling the font size.

[0048] The two outer points on the lower side are located at the intersection of the inscribed circle of the square and the crosshairs of the cross; while the two inner points on the lower side are the golden section points of the lines connecting the two outer points, which helps to make writing beautiful.

[0049] The four points below provide guidance for finishing the stroke;

[0050] All points are carefully arranged into a trapezoidal structure, outlining the solid framework of the Chinese character structure.

[0051] This layout not only reduces the difficulty of positioning in traditional calligraphy practice methods, but also deeply reflects the understanding and respect for the internal logic of Chinese character structure. In particular, in view of the common characteristics of Chinese characters, which are narrow at the top and wide at the bottom, and the left and right strokes are stretched, a more spacious space is creatively reserved between the two lower points.

[0052] In another preferred embodiment of the present invention, the process of performing direction correction on the written text image is as follows:

[0053] The edge lines of the writing paper area are obtained, and the slope differences between the four edge lines of the writing paper and the boundary lines of the writing text image are calculated respectively. The writing text image is offset corrected according to the average value of the slope differences.

[0054] In another preferred embodiment of the present invention, the process of detecting the distance between the upper and lower parts or the left and right parts of a character is as follows:

[0055] Traverse the text area according to a set direction, the set direction is a vertical direction or a horizontal direction, count the grayscale value of each column of pixels detected, the grayscale value of the text pixel is 0, the grayscale value of the pixel in the writing paper area is 255, select the first pixel column whose number of text pixels accounts for a ratio greater than a preset ratio k of the total number of pixels in the column, mark it as the start column of the text, continue to traverse the text area, when a pixel column whose number of text pixels accounts for a ratio less than k is detected, mark it as a pending column, obtain the pixel distance D1 between the pending column and the start column, if the pixel distance D1 is greater than or equal to the preset distance d1, and continue to detect along the set direction and no text pixels are found, mark the pending column as the end column, and the text is a monomer structure;

[0056] If the pixel distance is less than d, continue to traverse the text area. When a pixel column with a text pixel ratio greater than k is detected again, it is marked as the second starting column. The distance D2 between the second starting column and the pending column is obtained. If the distance D2 is greater than the preset distance d2, it is determined that the structure of the text is not standardized.

[0057] By traversing the text area in a set direction and counting the grayscale values ​​of each column of pixels, the system can accurately detect the start and end columns of the text. This method avoids the errors caused by subjective judgment in traditional methods and improves the accuracy of detection. Regardless of the vertical or horizontal arrangement of text, the system can perform adaptive detection according to the set direction. This flexibility allows the system to be suitable for students with different writing styles and habits, and has a wider range of applications.

[0058] In another preferred embodiment of the present invention, the specific process of determining whether the handwriting continuity of a text is insufficient is as follows:

[0059] For any selected grid, select the vertical line where the center point of the grid is located, obtain the distance of all pixels in the text relative to the vertical line, multiply the distance of each pixel point relative to the vertical line by the same ratio, and continue to increase the value of the ratio until adjacent grids are in contact with each other, to obtain a strip area of ​​text pixels, obtain the corresponding fitting straight line equation through median filtering of the strip area, and obtain the slope of the straight line equation.

[0060] By obtaining the distance of all pixels in the text relative to the vertical line and continuously increasing the proportional value until adjacent grids touch, the system can generate a continuous text pixel strip area. Then, by using median filtering to obtain the equation of the fitted line and calculate the slope, the system can accurately determine whether the handwriting continuity of the line of text is sufficient. This method avoids the errors caused by subjective judgment in traditional methods and improves the accuracy of evaluation.

[0061] In addition to judging the coherence of handwriting, the system can also conduct in-depth analysis of details such as handwriting fluency and strength. By accurately collecting and analyzing the starting and ending points of each stroke, the system can generate detailed handwriting feature graphics to provide students with comprehensive handwriting feedback.

[0062] In another preferred embodiment of the present invention, the process of generating the characteristic graph is:

[0063] First, select the three pending points that are farthest from the center point of the grid, connect the adjacent pending points to generate a pending feature graph, and then select the points with the farthest distance from the remaining pending points one by one to add to the lines of the feature graph, until the angle corresponding to the original feature point in the line formed by the newly added feature point becomes smaller, then the added point is extracted, and the line formed by the original feature point is the feature graph of the text.

[0064] The writing assistance method based on the cross-shaped grid and computer vision is applied to the above-mentioned writing assistance method based on the cross-shaped grid and computer vision, comprising the following steps:

[0065] Collecting written text images uploaded by users, and preprocessing the written text images to divide the written text images into a writing paper area and a background area;

[0066] Identify the edge of the writing paper area, and perform direction correction on the written text image according to the placement direction of the writing paper;

[0067] Select a text area in any grid on the writing paper, collect the starting point and ending point of each stroke based on the dot matrix smart pen, mark them as pending points, obtain several pending points farthest from the center point of the grid, define them as feature points, connect adjacent feature points in sequence to generate feature graphs, obtain the area of ​​the feature graph exceeding the preset positive grid area, and when the area is greater than a preset threshold, identify whether it is a left-right structure or an upper-lower structure. If not, it is judged that the writing of the text occupies a large grid; if so, detect the distance between the upper and lower parts or the left and right parts of the text. If the distance is greater than the preset distance, it is judged that the text structure is not standardized;

[0068] Select the text area of ​​any row, and horizontally enlarge the text area in each grid with the vertical line where the center point of the grid is located as the starting point until the text areas in adjacent grids touch each other, generate a continuous text pixel strip area, calculate the slope of the fitting line of the strip area, and when the difference between the slope and 0 is greater than the preset threshold, it is judged that the text handwriting of the row lacks continuity.

[0069] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A writing assistance system based on the cross-shaped grid and computer vision, characterized in that: include: An image acquisition module is used to acquire written text images uploaded by users, and pre-process the written text images to divide the written text images into a writing paper area and a background area; An image processing module, used for identifying the edge of the writing paper area and performing direction correction on the writing text image according to the placement direction of the writing paper; A single handwriting evaluation module is used to select a text area in any grid on the writing paper, collect the starting point and ending point of each stroke based on the dot matrix intelligent pen, mark them as pending points, obtain a number of pending points farthest from the center point of the grid, define them as feature points, connect adjacent feature points in sequence to generate a feature graph, obtain the area of ​​the feature graph exceeding the preset positive grid area, and when the area is greater than a preset threshold, identify whether it is a left-right structure or an upper-lower structure. If not, it is judged that the writing of the text occupies a large grid; if so, detect the distance between the upper and lower parts or the left and right parts of the text. If the distance is greater than the preset distance, it is judged that the text structure is not standardized; The coherent handwriting evaluation module is used to select the text area of ​​any line, horizontally enlarge the text area in each grid with the vertical line where the center point of the grid is located as the starting point, until the text areas in adjacent grids touch each other, generate a continuous text pixel strip area, calculate the slope of the fitting straight line of the strip area, and when the difference between the slope and 0 is greater than a preset threshold, it is judged that the text handwriting of the line is insufficient in coherence.

2. The writing assistive system based on the cross-shaped grid and computer vision according to claim 1, characterized in that: The pre-processing process is as follows: The written text image is grayscaled to generate a grayscale image, the contrast of the grayscale image is increased, and the grayscale image is converted into a binary image by setting a grayscale threshold. The binary image is denoised based on Gaussian filtering, and edge detection is performed on the denoised binary image to segment the writing paper area from the background area.

3. The writing assistive system based on the cross-shaped grid and computer vision according to claim 1, characterized in that: The preset positive ladder grid area is composed of 6 calibration points, 2 of which are set in the upper half of the grid, and the other 4 points are set in the lower half of the grid. The two points in the upper half are located at the midpoints of the oblique lines of the left and right upper half grids; two of the points in the lower half are located at the intersection of the inscribed circle of the square grid and the oblique line of the grid, that is, the outer points, and the remaining two points in the lower half are located at the two golden section points of the line connecting the two outer points.

4. The writing assistive system based on the cross-shaped grid and computer vision according to claim 1, characterized in that: The process of correcting the direction of a written text image is as follows: The edge lines of the writing paper area are obtained, and the slope differences between the four edge lines of the writing paper and the boundary lines of the writing text image are calculated respectively. The writing text image is offset corrected according to the average value of the slope differences.

5. The writing assistive system based on the cross-shaped grid and computer vision according to claim 1, characterized in that: The process of detecting the distance between the upper and lower parts or the left and right parts of a text is: Traverse the text area according to a set direction, the set direction is a vertical direction or a horizontal direction, count the grayscale value of each column of pixels detected, the grayscale value of the text pixel is 0, the grayscale value of the pixel in the writing paper area is 255, select the first pixel column whose number of text pixels accounts for a ratio greater than a preset ratio k of the total number of pixels in the column, mark it as the start column of the text, continue to traverse the text area, when a pixel column whose number of text pixels accounts for a ratio less than k is detected, mark it as a pending column, obtain the pixel distance D1 between the pending column and the start column, if the pixel distance D1 is greater than or equal to the preset distance d1, and continue to detect along the set direction and no text pixels are found, mark the pending column as the end column, and the text is a monomer structure; If the pixel distance is less than d, continue to traverse the text area. When a pixel column with a text pixel ratio greater than k is detected again, it is marked as the second starting column. The distance D2 between the second starting column and the pending column is obtained. If the distance D2 is greater than the preset distance d2, it is determined that the structure of the text is not standardized.

6. The writing assistive system based on the cross-shaped grid and computer vision according to claim 5, characterized in that: The specific process of judging the lack of coherence of handwriting is as follows: For any selected grid, select the vertical line where the center point of the grid is located, obtain the distance of all pixels in the text relative to the vertical line, multiply the distance of each pixel point relative to the vertical line by the same ratio, and continue to increase the value of the ratio until adjacent grids are in contact with each other, to obtain a strip area of ​​text pixels, obtain the corresponding fitting straight line equation through median filtering of the strip area, and obtain the slope of the straight line equation.

7. The writing assistive system based on the cross-shaped grid and computer vision according to claim 1, characterized in that: The process of generating feature graphs is: First, select the three pending points that are farthest from the center point of the grid, connect the adjacent pending points to generate a pending feature graph, and then select the points with the farthest distance from the remaining pending points one by one to add to the lines of the feature graph, until the angle corresponding to the original feature point in the line formed by the newly added feature point becomes smaller, then the added point is extracted, and the line formed by the original feature point is the feature graph of the text.

8. A writing assistance method based on a grid and computer vision, applied to the writing assistance method based on a grid and computer vision as claimed in any one of claims 1 to 7, characterized in that: The following steps are involved: Collecting written text images uploaded by users, and preprocessing the written text images to divide the written text images into a writing paper area and a background area; Identify the edge of the writing paper area, and perform direction correction on the written text image according to the placement direction of the writing paper; Select a text area in any grid on the writing paper, collect the starting point and ending point of each stroke based on the dot matrix smart pen, mark them as pending points, obtain several pending points farthest from the center point of the grid, define them as feature points, connect adjacent feature points in sequence to generate feature graphs, obtain the area of ​​the feature graph exceeding the preset positive grid area, and when the area is greater than a preset threshold, identify whether it is a left-right structure or an upper-lower structure. If not, it is judged that the writing of the text occupies a large grid; if so, detect the distance between the upper and lower parts or the left and right parts of the text. If the distance is greater than the preset distance, it is judged that the text structure is not standardized; Select the text area of ​​any row, and horizontally enlarge the text area in each grid with the vertical line where the center point of the grid is located as the starting point until the text areas in adjacent grids touch each other, generate a continuous text pixel strip area, calculate the slope of the fitting line of the strip area, and when the difference between the slope and 0 is greater than the preset threshold, it is judged that the text handwriting of the row lacks continuity.

Images