Handwritten Chinese character automatic perspective correction and region extraction method and system

Abstract

The invention provides a handwritten Chinese character automatic perspective correction and region extraction method and system. The method is applied to the technical field of computer vision and image processing, and comprises the following steps: collecting handwritten Chinese character copybooks and Chinese character copybook images, and carrying out size and color processing on the images to obtain an image data set; processing the processed image by adopting a YOLO detection method to obtain a copybook bounding box, and performing perspective transformation correction on the bounding box; carrying out corner detection on the pre-corrected copybook by adopting an improved Harris method, and extracting to obtain image corner image features; and carrying out accurate region extraction and optimization on the basis of corner image features obtained after detection to complete region extraction. In this way, the Chinese character image correction precision is effectively improved, and the method has the advantages of being high in processing speed, high in automation degree and small in occupied resource and has high robustness.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of computer vision and image processing, and in particular to a method and system for automatic perspective correction and region extraction of handwritten Chinese characters. Background Art

[0002] In recent years, with the popularization of mobile devices and the increasing demand for office automation, handwritten text image processing technology has developed rapidly. Traditional text recognition technology is mainly aimed at printed text, but the processing difficulty of handwritten text, especially Chinese characters with rich stroke features, is significantly increased. There are currently a variety of handwritten text processing solutions on the market, including methods based on traditional image processing and methods based on deep learning. However, these solutions still have many problems in practical applications. Mainly due to factors such as shooting angle, lighting conditions, writing style, and complex strokes, there are often problems such as blurred image acquisition, difficulty in feature extraction, and low processing efficiency, which greatly increases the difficulty of extracting printed text such as Chinese characters.

[0003] Therefore, there is an urgent need for a Chinese character image processing design solution that can effectively ensure the accuracy and efficiency of Chinese character extraction processing. Summary of the Invention

[0004] The present invention provides a method and system for automatic perspective correction and region extraction of handwritten Chinese characters, which realizes high-precision Chinese character region extraction through overall detection of copybooks, perspective correction, and internal structure analysis.

[0005] According to a first aspect of the present disclosure, a method for automatic perspective correction and region extraction of handwritten Chinese characters is provided, comprising the following steps: Collect handwritten Chinese character copybooks and collect Chinese character copybook images. After size and color processing of the images, an image dataset is obtained. The processed image is processed using the YOLO detection method to obtain the copybook bounding box, and the bounding box is corrected for perspective transformation; The improved Harris method is used to detect corners of the pre-corrected copybook and extract image corner features. Based on the corner image features obtained after detection, accurate region extraction and optimization are performed to complete region extraction.

[0006] According to the above aspect and any possible implementation, a further implementation is provided, wherein the process of resizing the image is as follows: Input an original image of a Chinese character copybook, and calculate the original height and width of the original image; Set the target size, and calculate the aspect ratio factor based on the original image height and width, and use the smaller of the two ratio factors as the scaling factor; Based on the scaling factor, scaling the original image using an improved bilinear interpolation method to obtain a scaled image; A fixed value is calculated based on minimizing boundary effects, and the scaled image is filled with the fixed value.

[0007] According to the above aspects and any possible implementation, an implementation is further provided, wherein the specific process of the color processing is: Based on the human eye perception characteristics, the grayscale value of the original image is calculated; An average brightness value is calculated based on the original image, and a γ value is calculated using an adaptive method of analyzing an image histogram; Correcting the image pixel values ​​based on the γ value and using an adaptive gamma correction mechanism to obtain a corrected image; The improved two-dimensional Gaussian kernel function is used to smooth the rectified image.

[0008] According to the above aspects and any possible implementation, an implementation is further provided, wherein the improved two-dimensional Gaussian kernel function is specifically: ; Where G(x,y) represents the value of the two-dimensional Gaussian kernel function at the point (x,y), and σ is the standard deviation of the Gaussian function.

[0009] According to the above aspects and any possible implementation, an implementation is further provided, wherein the process of using the YOLO detection method to process the processed image, obtaining the copybook bounding box, and performing perspective transformation correction on the bounding box is as follows: The robust estimation method based on RANSAC is used to calculate the homography matrix based on the projection relationship of corresponding point pairs; Based on the homography matrix, optimizing the bounding box geometry constraints using parallelism constraints, orthogonality constraints, area ratio constraints, and projective invariant constraints; The Levenberg-Marquardt method is used for refinement iteration, and the boundary is processed by the reprojection method to complete the perspective transformation correction.

[0010] According to the above aspects and any possible implementation, a further implementation is provided, wherein the process of using the improved Harris method to perform corner detection on the pre-corrected copybook and extracting image corner features is as follows: The image of the pre-corrected copybook is filtered using a differential operator, and the difference of each pixel in the image is calculated. 、 as well as ,in, I x for x The gradient of the direction, I y for y Directional gradient; right 、 as well as Perform Gaussian smoothing to remove image noise; Calculate the response function of each corresponding point in the original image, enhance the response function by introducing a directional weighting factor, and perform local non-maximum suppression on the enhanced response function; A threshold is set based on the RANSAC method, and if the enhanced response function is greater than the threshold and is a local maximum in the domain, the point is extracted as a corner point; The extracted corner points are optimized using a multi-level corner point optimization method to obtain optimized corner point features.

[0011] In accordance with the above aspects and any possible implementation, a further implementation is provided, wherein the accurate region extraction and optimization are performed based on the corner image features obtained after detection, and the process of completing the region extraction is as follows: Construct a region growing method that combines multiple features and performs multi-scale feature extraction to complete image segmentation; Perform multi-scale feature extraction on the segmented image features and segment them by constructing a semantic segmentation network based on the U-Net structure to obtain the segmented image; Perform hierarchical local analysis on the segmented image to obtain the extraction results, and then optimize the extraction results to complete the region extraction.

[0012] According to the above aspects and any possible implementation, a further implementation is provided, wherein the process of optimizing the extraction results is as follows: Based on the regional merging criterion, the extracted results are merged and segmented to obtain the processed results; constructing an active contour energy function, and performing boundary smoothing on the processed result based on the function; The optimization process is completed by constructing regional feature vectors and filtering the noise area.

[0013] According to a second aspect of the present disclosure, there is provided a handwritten Chinese character automatic perspective correction and region extraction system, comprising: an image acquisition and preprocessing module, a perspective transformation correction module, a corner point detection module, and a region extraction module; The image acquisition and preprocessing module is used to collect handwritten Chinese character copybooks and acquire Chinese character copybook images, and obtain image data sets after size and color processing of the images; The perspective transformation correction module is used to process the processed image using the YOLO detection method, obtain the copybook boundary frame, and perform perspective transformation correction on the boundary frame; The corner detection module is used to perform corner detection on the pre-corrected copybook using an improved Harris method to extract image corner features; The region extraction module is used to perform accurate region extraction and optimization based on the corner point image features obtained after detection, thereby completing region extraction.

[0014] Compared with the prior art, the present invention has the following technical effects: (1) The present invention adopts a YOLO-based copybook detection technology: an improved YOLOv8 model structure is specially optimized for copybook features, an innovative data enhancement strategy is adopted to improve detection robustness, and an optimized loss function design is adopted to improve positioning accuracy; (2) The present invention adopts an accurate perspective transformation method: by estimating the homography matrix based on RANSAC, combining geometric constraints, and introducing projective invariant constraints, the geometric consistency of the transformation is ensured, and a boundary condition processing mechanism is designed to improve the correction effect; (3) The present invention adopts an improved Harris corner detection algorithm: by introducing a directional weighting factor, the sensitivity to the stroke features of Chinese characters is improved, combined with the design of a direction-adaptive suppression window, it can process multi-directional features, and adopts a significance scoring mechanism to improve detection accuracy.

[0015] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which: Figure 1 A schematic flow chart of a method for automatic perspective correction and region extraction of handwritten Chinese characters according to an embodiment of the present disclosure is shown; Figure 2 A schematic structural diagram of a handwritten Chinese character automatic perspective correction and region extraction system according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0017] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Reference Figure 1 As shown, this embodiment provides a method for automatic perspective correction and region extraction of handwritten Chinese characters, comprising the following steps: S101, collecting handwritten Chinese character copybooks and acquiring Chinese character copybook images, and obtaining image data sets after size and color processing of the images.

[0020] In this embodiment, after collecting the Chinese character copybook image, the image size is adjusted using the intelligent size adjustment method. The specific process is as follows: First, the original size of the input image is analyzed, combined with the target size of 640×640 pixels, and the width and height scaling factors are calculated, and the smaller scaling factor is selected as the scaling factor.

[0021] Zoom factor The calculation formula is: (1); Among them, W_target and H_target are the target width and height respectively, and W_original and H_original are the width and height of the original image respectively.

[0022] Secondly, the bilinear interpolation algorithm is used to scale the image. During the scaling process, an improved bilinear interpolation algorithm is used. For the target pixel point (x, y), the calculation formula for its pixel value f(x, y) is: (2); Where f(x, y) represents the interpolation result at the target position (x, y), f(x_i, y_j) represents the value of the four adjacent pixels, i,j ∈ {0,1} represents the adjacent 2×2 pixel area, and w(i, j) is the weight coefficient, which is calculated as follows: (3); Traditional bilinear interpolation is based on linear weighting of distance, but in this implementation, an improved weight calculation method shown in formula (3) is used. This calculation method takes into account the relative distance in both the x and y directions, making the interpolation result smoother and more accurate.

[0023] Finally, a fixed value is used for filling: For the area that needs to be filled, a fixed value (114, 114, 114) is used for filling. This value is the optimal solution obtained by minimizing the boundary effect.

[0024] Meanwhile, in this embodiment, the specific process of the color space conversion method is as follows: First, based on the human eye perception characteristics, an optimized weight coefficient Y, that is, the grayscale value, is designed. The calculation formula of the grayscale value Y is: (4); Among them, R, G, and B are the red, green, and blue component values ​​of the pixel respectively.

[0025] Equation 4 shows the optimized weighting system, which is designed based on the perceptual characteristics of the human eye. The weighting system of this embodiment reflects the sensitivity of the human eye to different colors. Specifically, the human eye is most sensitive to green (0.587), followed by red (0.299), and least sensitive to blue (0.114). The sum of these coefficients is equal to 1, ensuring the correct calculation of the brightness value.

[0026] Secondly, an adaptive gamma correction mechanism is introduced. The calculation formula of the corrected output image pixel value I' is: (5); in, is the scaling factor used to adjust the overall brightness, I is the pixel value of the input image, and γ is the correction coefficient, which is adaptively calculated by analyzing the image histogram: (6); Among them, L_mean is the average brightness value of the image.

[0027] Finally, the improved two-dimensional Gaussian kernel function G(x,y) is used for image smoothing: (7); Where G(x,y) represents the value of the two-dimensional Gaussian kernel function at the point (x,y), and σ is the standard deviation of the Gaussian function, which determines the width of the Gaussian kernel. A larger σ value results in a wider Gaussian kernel and a more pronounced smoothing effect; a smaller σ value results in a narrower Gaussian kernel and a weaker smoothing effect. In practical applications, the value of σ is typically adjusted adaptively based on the noise level of the image: (8); in, is the scaling factor, noise_level is estimated through local variance, and noise_level is the noise level estimated through local variance. Equation 8 indicates that the standard deviation of the Gaussian kernel is adaptively adjusted according to the noise level of the image.

[0028] This embodiment also improves the traditional binarization method, specifically: First, the inter-class variance weighting mechanism is introduced, and the calculation formula of the optimal threshold t* is: (9); in, and are the weights of foreground and background respectively, and are the average gray values ​​of foreground and background, respectively.

[0029] Secondly, for each pixel (x, y) in the image, the calculation formula of its local threshold T(x, y) is: (10); Among them, m(x,y) is the local mean, s(x,y) is the local standard deviation, is the global standard deviation, is the adjustment parameter.

[0030] S102: Use the YOLO detection method to process the processed image, obtain the copybook boundary box, and perform perspective transformation correction on the boundary box.

[0031] The present invention adopts an innovative matrix calculation method: First, robust estimation based on RANSAC: The calculation of the homography matrix H is based on the projection relationship of corresponding point pairs: (11); Where x and x' are the homogeneous coordinates of the corresponding points.

[0032] The calculation of H uses the improved RANSAC algorithm, and the consistency score Q is calculated as: (12); Where d is the reprojection error and δ is the scale parameter.

[0033] Secondly, geometric constraint optimization: introduce multiple geometric constraints: Parallelism constraints: ; Orthogonality constraints: ; Area ratio constraint: (13); Among them, l1 and l2 represent two straight lines in the original image, l1' and l2' represent the corresponding straight lines in the transformed image, and H T is the transpose of the homography matrix, A represents the area in the original image, A' represents the area after transformation, and det(H) represents the determinant value of the homography matrix H.

[0034] Finally, the iterative refinement strategy is based on the Levenberg-Marquardt algorithm, and the optimization objective function is: (14); Among them, w_i is the weight coefficient, R(H) is the regularization term, λ is the balance parameter, Represents the square of the Euclidean distance between corresponding points before and after the transformation.

[0035] At the same time, the present invention designs a parameter optimization mechanism: First, geometric invariances are preserved: constraints based on projective invariants: Alternating invariant ratio: ; Collinearity is maintained: (15); Where A, B, C, and D represent the four collinear points in the original image, A', B', C', and D' represent the four points whose cross-ratios remain unchanged under the projection transformation after transformation, det([x1x2x3]) = 0, and x1, x2, and x3 represent the homogeneous coordinates of the three points. When det = 0, it means that x1, x2, and x3 are collinear.

[0036] Secondly, the distortion compensation mechanism: radial distortion model: (16); (17); Tangential distortion model: (18); (19); Among them, (x,y) is the ideal coordinate, (x_d,y_d) is the actual coordinate, , k1, k2 are radial distortion coefficients, p1, p2 are tangential distortion coefficients.

[0037] Finally, boundary condition processing: using the interpolation algorithm with directional derivative constraints: (20); Where n is the direction of the boundary normal vector, is the image intensity. Equation 20 indicates that the normal derivative of the image intensity is zero at the boundary, that is, the boundary has a smooth transition.

[0038] Finally, the present invention adopts a high-quality reprojection method: First, improved bilinear interpolation: adaptive weight calculation based on directional derivatives: (twenty one); in, is the image gradient, is the interpolation direction vector.

[0039] Secondly, edge pixel processing: interpolation weight adjustment of edge area: (twenty two); Among them, edge(i,j) is the edge strength, is the enhancement factor.

[0040] S103, using the improved Harris method to perform corner detection on the pre-corrected copybook, and extracting image corner features.

[0041] The specific process is: using the differential operator to filter the image of the pre-corrected copybook, and calculating the 、 as well as , where I x is the gradient in the x direction, I y is the gradient in the y direction; right 、 as well as Perform Gaussian smoothing to remove image noise; Calculate the response function of each corresponding point in the original image, enhance the response function by introducing a directional weighting factor, and perform local non-maximum suppression on the enhanced response function; A threshold is set based on the RANSAC method, and if the enhanced response function is greater than the threshold and is a local maximum in the domain, the point is extracted as a corner point; The extracted corner points are optimized using a multi-level corner point optimization method to obtain optimized corner point features.

[0042] Specifically, the present invention comprehensively improves the traditional Harris algorithm: First, in the scale space, the Harris response function R(x,y,σ) is calculated as: (twenty three); Where M is the Harris matrix: ; Among them, I_x and I_y are the image gradients in the x and y directions respectively, G_σ is the Gaussian kernel function, is an empirical constant (usually 0.04-0.06).

[0043] Secondly, the directional weighting factor w(θ) is introduced: (twenty four); Among them, θ is the gradient direction of the current pixel, θ_dominant is the local main direction, is the weight coefficient.

[0044] The enhanced Harris response function is: (25); Finally, based on the RANSAC idea, the calculation formula of the threshold T is: (26); Among them, μ_R and σ_R are the mean and standard deviation of the response values, respectively. Confirmed by consistency check.

[0045] The present invention adopts a multi-level corner point optimization method: First, a direction-adaptive suppression window W(x,y,θ) is designed: (27); Where r is the window radius, is the local gradient direction, (u, v) is the coordinate of any point in the window, Control the horizontal window size, Control the vertical window size, |ux|≤r· represents the horizontal constraint, |vy|≤r· Represents a constraint in the vertical direction.

[0046] Secondly, the screening mechanism based on significance: the corner point significance score S is calculated as follows: (28); in, is the Harris response value, C is the local contrast, and D is the spatial distribution score. 、 、 is the weight coefficient.

[0047] Finally, based on the Delaunay triangulation, the spatial distribution uniformity measure E is defined: (29); Among them, A_i is the area of ​​the triangle, A_mean is the average area, is the number of triangles.

[0048] S104: Perform accurate region extraction and optimization based on the corner image features obtained after detection to complete region extraction.

[0049] The present invention designs an accurate regional positioning method: First, the improved connected domain analysis: region growing criterion: (30); Where T(p,q) is the adaptive threshold, I(p) is the grayscale value of the seed point p, and I(q) is the grayscale value of the candidate point q: (31); Among them, σ_local is the local standard deviation, is the proportional coefficient used to adjust the sensitivity of the threshold, and c is the constant offset used to provide the basic threshold.

[0050] Secondly, hierarchical region growing: a growth criterion based on multi-feature fusion: (32); Among them, D_intensity is the grayscale difference, D_gradient is the gradient difference, D_texture is the texture difference, and D(p,q) is the comprehensive difference metric. is the grayscale difference weight, is the gradient difference weight, is the texture difference weight.

[0051] Finally, adaptive segmentation threshold: local contrast adaptation mechanism: (33); Among them, μ_local is the local mean, reflecting the average brightness level of the area, σ_local is the local area standard deviation, reflecting the degree of change in the area, T(x,y) is the adaptive threshold, is the local coefficient.

[0052] The present invention adopts advanced layout analysis technology: First, multi-scale feature extraction: feature pyramid construction: (34); Among them, P_l is the pooling operation, l is the pyramid level, and F_I is the feature value of different levels.

[0053] Secondly, semantic segmentation network: loss function based on U-Net structure: (35); Among them, L_ce is the cross entropy loss, L_dice is the Dice coefficient loss, L_boundary is the boundary loss, λ1 and λ2 are the weight coefficients of the loss function, which are used to balance the contribution of different loss terms.

[0054] Finally, hierarchical layout analysis: Similarity measurement of document structure tree construction: (36); Among them, sim_spatial is the spatial similarity, sim_semantic is the semantic similarity, , is the weight coefficient, and + = 1.

[0055] The present invention optimizes the extraction results by the following steps: First, region merging and segmentation: Region merging criteria: ,if (37); in, is the distance measure between regions, For the region and Merge operation, T_merge is the merge threshold, , The area to be merged.

[0056] (38); Among them, D_color is the color feature distance, D_texture is the texture feature distance, and D_shap is the shape feature distance. , , are the weight coefficients of each feature, and + + = 1.

[0057] Secondly, boundary smoothing: active contour energy function: (39) Among them, E_internal is the internal energy, E_external is the external energy, E_constraint is the constraint energy, and α, β, and γ are the weight coefficients of the energy terms.

[0058] Finally, noise area filtering: construction of regional feature vector f: (40); Among them, f_size is the region size feature, f_shape is the region shape feature, f_texture is the region texture feature, and f_location is the region location feature.

[0059] Noise area determination criteria: ,if (41); Among them, is_noise( ) is the judgment area Whether it is noise, T_noise is the noise judgment threshold, and score(f) is the scoring function based on multi-feature fusion.

[0060] like Figure 2 As shown, this embodiment also provides a handwritten Chinese character automatic perspective correction and region extraction system, including: an image acquisition and preprocessing module 1, a perspective transformation correction module 2, a corner point detection module 3 and a region extraction module 4; The image acquisition and preprocessing module 1 is used to collect handwritten Chinese character copybooks and acquire Chinese character copybook images, and obtain image data sets after size and color processing of the images; The perspective transformation correction module 2 is used to process the processed image using the YOLO detection method, obtain the copybook boundary frame, and perform perspective transformation correction on the boundary frame; The corner detection module 3 is used to detect corners of the pre-corrected copybook using the improved Harris method, and extract image corner features; The region extraction module 4 is used to perform accurate region extraction and optimization based on the corner point image features obtained after detection, thereby completing region extraction.

[0061] This invention utilizes a multi-level optimization strategy to enhance computational efficiency. For parallel computing, it fully leverages the parallel processing capabilities of modern computing hardware to implement a multi-level parallel computing framework. At the CPU level, the OpenMP parallel computing framework is employed, enabling multi-threaded parallel execution of image processing tasks through the #pragma omp parallel directive. For computationally intensive tasks such as corner detection and feature extraction, thread pooling technology is employed to achieve dynamic task scheduling and load balancing.

[0062] This invention also supports GPU-accelerated computing, implementing GPU parallelization of key algorithms based on the CUDA framework. By optimizing kernel function design and leveraging the GPU's SIMD (Single Instruction, Multiple Data) capabilities, it significantly improves the efficiency of large-scale data processing. In actual tests, the GPU-accelerated version achieved a 3-5x performance improvement over the CPU version when processing large images.

[0063] In terms of algorithm optimization, this paper employs a series of optimization strategies to improve computational efficiency while maintaining accuracy. For computationally intensive operations, such as corner response function calculations, a fast approximation algorithm based on Taylor expansion is employed. By finding a balance between accuracy and efficiency, the computational effort is significantly reduced while maintaining an acceptable accuracy loss (error controlled within 1%).

[0064] This paper designs and implements an efficient memory pool management system to optimize frequent memory allocation and deallocation operations. The memory pool utilizes a layered design, comprising large and small object pools to handle memory requests of varying sizes. A recycling strategy is employed to allocate temporary buffers, a common problem in image processing, significantly reducing memory fragmentation and allocation overhead.

[0065] In terms of cache optimization, we implemented a multi-level cache optimization strategy based on the characteristics of modern computer architectures. By analyzing the program's data access patterns, we optimized the memory layout of data structures and improved cache hit rates. We also employed data alignment, prefetching mechanisms, locality optimization, and cache partitioning to effectively enhance system performance.

[0066] This paper implements a complete exception handling framework to ensure the system can operate stably under various abnormal conditions. The exception handling mechanism includes comprehensive validity checks of input data, status monitoring and anomaly detection of key steps, multi-level error recovery strategies, and a hierarchical logging system.

[0067] In terms of adaptive optimization, the system can dynamically adjust parameters according to input characteristics and operating environment, implement task scheduling strategies based on performance feedback, and establish a complete performance indicator monitoring and analysis system.

[0068] In specific applications, the present invention can be applied to the following scenarios: 1. Document Scanning Application: In office document digitization scenarios, this system can effectively handle the scanning needs of various paper documents. The system can automatically correct document perspective distortion, accurately extract text areas, and optimize the output results to ensure the quality of digitized documents.

[0069] In document recognition applications, the system can accurately locate the position of the identification document, precisely extract the document border, reasonably segment the text area, and improve the recognition effect through image enhancement technology.

[0070] 2. Educational Applications: In the homework grading scenario, the system can automatically correct test paper images, accurately extract the answer area, identify the teacher's annotation area, and support statistical analysis of grades. This greatly improves teachers' work efficiency.

[0071] For class note processing, the system can capture the content on the blackboard, correct the deformation of the projected content, extract key note areas, and improve the readability of the content through image enhancement technology.

[0072] 3. Mobile application scenarios: In mobile scanning applications, the system provides real-time preview and correction functions, supports fast image capture, can automatically perform cropping optimization, and complete image processing locally, providing a good user experience.

[0073] In terms of document management, the system supports batch processing of documents, provides an intelligent classification storage mechanism, supports content-based retrieval functions, and can synchronize with cloud services to ensure data security.

[0074] It can be seen that the correction accuracy of the present invention is high: the average error of perspective transformation is less than 1%, which greatly improves the accuracy of image correction; the processing speed is fast: real-time processing can be achieved on mainstream hardware platforms, with an average processing time of less than 100ms; it is robust: it can adapt to various shooting conditions and document types, with a success rate of more than 95%; it has a high degree of automation: no human intervention is required, and it can be fully automated; it takes up little resources: it takes up little memory (<500MB), and is suitable for mobile device applications.

[0075] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0076] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0077] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for automatic perspective correction and region extraction of handwritten Chinese characters, characterized in that: The following steps are involved: Collect handwritten Chinese character copybooks and collect Chinese character copybook images. After size and color processing of the images, an image dataset is obtained. The processed image is processed using the YOLO detection method to obtain the copybook bounding box, and the bounding box is corrected for perspective transformation; The improved Harris method is used to detect corners of the pre-corrected copybook and extract image corner features. Based on the corner image features obtained after detection, accurate region extraction and optimization are performed to complete region extraction.

2. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 1, characterized in that: The process of resizing the image is as follows: Input an original image of a Chinese character copybook, and calculate the original height and width of the original image; Set the target size, and calculate the aspect ratio factor based on the original image height and width, and use the smaller of the two ratio factors as the scaling factor; Based on the scaling factor, scaling the original image using an improved bilinear interpolation method to obtain a scaled image; A fixed value is calculated based on minimizing boundary effects, and the scaled image is filled with the fixed value.

3. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 1, characterized in that: The specific process of the color processing is: Based on the human eye perception characteristics, the grayscale value of the original image is calculated; An average brightness value is calculated based on the original image, and a γ value is calculated using an adaptive method of analyzing an image histogram; Correcting the image pixel values ​​based on the γ value and using an adaptive gamma correction mechanism to obtain a corrected image; The improved two-dimensional Gaussian kernel function is used to smooth the rectified image.

4. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 3, characterized in that: The improved two-dimensional Gaussian kernel function is specifically: ； Where G(x,y) represents the value of the two-dimensional Gaussian kernel function at the point (x,y), and σ is the standard deviation of the Gaussian function.

5. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 1, characterized in that: The process of using the YOLO detection method to process the processed image, obtain the copybook bounding box, and perform perspective transformation correction on the bounding box is as follows: The robust estimation method based on RANSAC is used to calculate the homography matrix based on the projection relationship of corresponding point pairs; Based on the homography matrix, optimizing the bounding box geometry constraints using parallelism constraints, orthogonality constraints, area ratio constraints, and projective invariant constraints; The Levenberg-Marquardt method is used for refinement iteration, and the boundary is processed by the reprojection method to complete the perspective transformation correction.

6. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 1, characterized in that: The process of using the improved Harris method to detect corners of the pre-corrected copybook and extracting image corner features is as follows: The image of the pre-corrected copybook is filtered using a differential operator, and the difference of each pixel in the image is calculated. 、 as well as ,in, I x is the gradient in the x direction, I y is the gradient in the y direction; right 、 as well as Perform Gaussian smoothing to remove image noise; Calculate the response function of each corresponding point in the original image, enhance the response function by introducing a directional weighting factor, and perform local non-maximum suppression on the enhanced response function; A threshold is set based on the RANSAC method, and if the enhanced response function is greater than the threshold and is a local maximum in the domain, the point is extracted as a corner point; The extracted corner points are optimized using a multi-level corner point optimization method to obtain optimized corner point features.

7. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 1, characterized in that: The process of performing accurate region extraction and optimization based on the corner image features obtained after detection and completing region extraction is as follows: Construct a region growing method that combines multiple features and performs multi-scale feature extraction to complete image segmentation; Perform multi-scale feature extraction on the segmented image features and segment them by constructing a semantic segmentation network based on the U-Net structure to obtain the segmented image; Perform hierarchical local analysis on the segmented image to obtain the extraction results, and then optimize the extraction results to complete the region extraction.

8. The method for automatic perspective correction and region extraction of handwritten Chinese characters according to claim 7, characterized in that: The process of optimizing the extraction results is as follows: Based on the regional merging criterion, the extracted results are merged and segmented to obtain the processed results; constructing an active contour energy function, and performing boundary smoothing on the processed result based on the function; The optimization process is completed by constructing regional feature vectors and filtering the noise area.

9. A system for automatic perspective correction and region extraction of handwritten Chinese characters, for implementing the method for automatic perspective correction and region extraction of handwritten Chinese characters according to any one of claims 1 to 8, characterized in that: include: Image acquisition and preprocessing module, perspective transformation correction module, corner detection module and region extraction module; The image acquisition and preprocessing module is used to collect handwritten Chinese character copybooks and acquire Chinese character copybook images, and obtain image data sets after size and color processing of the images; The perspective transformation correction module is used to process the processed image using the YOLO detection method, obtain the copybook boundary frame, and perform perspective transformation correction on the boundary frame; The corner detection module is used to perform corner detection on the pre-corrected copybook using an improved Harris method to extract image corner features; The region extraction module is used to perform accurate region extraction and optimization based on the corner point image features obtained after detection, thereby completing region extraction.

Images