An optical character recognition method for assisting teaching

Abstract

The application discloses an optical character recognition method for assisting teaching and belongs to the technical field of image processing. The application first collects a handwritten text image and divides a single character image, then obtains direction intensity of 8 direction intervals according to a gradient direction of a pixel point of the character image, determines a direction dominant degree by using a ratio of maximum direction intensity and total direction intensity, and obtains a direction dominant degree distribution image; meanwhile, direction mutation intensity of each pixel point is calculated to generate a direction mutation intensity distribution image. Subsequently, feature enhancement is performed on the character image, the direction dominant degree distribution image and the direction mutation intensity distribution image, and stroke feature maps and turning feature maps are extracted. Finally, multi-level feature extraction and fusion are performed on the two types of feature maps, full connection layers are used for completing classification and recognition, and an identification result is output, so that the precision of handwritten text optical character recognition is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an optical character recognition method for assisting teaching. Background Technology

[0002] With the accelerated advancement of educational informatization, the demand for digitizing students' handwritten text in classroom teaching and homework correction is becoming increasingly prominent. Quickly and accurately converting students' handwritten text into editable electronic text can significantly improve teachers' teaching and correction efficiency, and also provide strong data support for subsequent learning analysis and teaching resource integration. However, the application of handwritten text recognition technology in actual teaching scenarios still faces many challenges, with insufficient recognition accuracy being a key bottleneck restricting its widespread application.

[0003] Currently, the technical solution for image-based text recognition mainly combines image processing and deep learning.

[0004] While deep learning-based image recognition technology has improved recognition performance to some extent, it still has limitations. Most deep learning models focus on extracting overall features from images. The specific process includes filtering the image of handwritten text and then using a convolutional neural network (CNN) to recognize the filtered image. However, this method does not capture the stroke features and transition features in handwritten text in sufficient detail, resulting in low recognition accuracy. Summary of the Invention

[0005] In view of the above-mentioned shortcomings in the prior art, the present invention provides an optical character recognition method for auxiliary teaching, which solves the problem of low recognition accuracy in the prior art.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: an optical character recognition method for assisting teaching, comprising the following steps:

[0007] Collect images of handwritten text and segment individual characters from the images to obtain character images;

[0008] Based on the gradient direction of each character pixel in the character image, the directional intensity of 8 directional intervals is obtained;

[0009] The ratio of the maximum directional intensity to the total directional intensity of the same character pixel is used as the directional dominance, and the directional dominance distribution image is obtained.

[0010] Calculate the intensity of directional abrupt changes for each character pixel in the character image to obtain an image of the distribution of directional abrupt changes intensity.

[0011] Feature enhancement was performed on the character image, the orientation dominance distribution image, and the orientation change intensity distribution image, and stroke feature maps and transition feature maps were extracted respectively.

[0012] Multi-level feature extraction and fusion were performed on the stroke feature map and the transition feature map respectively, and a fully connected layer was used for classification and recognition to obtain the recognition results.

[0013] Furthermore, the process of obtaining the directional intensity of the eight directional intervals includes:

[0014] The horizontal gradient component of each character pixel in the character image is calculated using the horizontal gradient operator in the Prewitt operator.

[0015] The vertical gradient component of each character pixel in the character image is calculated using the vertical gradient operator in the Prewitt operator.

[0016] The gradient direction is obtained from the horizontal and vertical gradient components.

[0017] Extending the gradient direction to the range of [0°, 360°] yields the extended direction;

[0018] Divide [0°, 360°] into 8 directional intervals;

[0019] Using each character pixel as the center, the directional intensity is calculated based on the expansion direction of all character pixels within the same directional interval in the neighborhood of the center.

[0020] Furthermore, the formula for calculating directional intensity is:

[0021] ,

[0022] Among them, E θ,i,d Let θ be the directional intensity of the d-th direction interval of the i-th text pixel. i,j Let || represent the expansion direction of the j-th character pixel within the central neighborhood when the i-th character pixel is the center, || represents the absolute value, d is the index of the direction interval, and d is a positive integer ranging from 0 to 7, N is the number of character pixels within the central neighborhood, i and j are positive integers, and cos is the cosine function.

[0023] Furthermore, the process of obtaining the orientation dominance distribution image includes:

[0024] The total directional intensity is obtained by summing the directional intensities of the eight directional intervals for each character pixel.

[0025] Select the maximum directional intensity from the directional intensities of the eight directional intervals for each text pixel;

[0026] The ratio of maximum directional intensity to total directional intensity is used as the directional dominance of the corresponding text pixel.

[0027] Arrange the directional dominance values ​​according to the corresponding text pixels, and set the directional dominance values ​​of other background pixels to 0 to obtain the directional dominance distribution image.

[0028] Furthermore, the process of obtaining the intensity distribution image of directional mutations includes:

[0029] For each character pixel in the character image, the direction interval corresponding to the maximum directional intensity is taken as the dominant direction;

[0030] Calculate the directional difference between each text pixel and its neighboring text pixels in the dominant direction;

[0031] The mean of the directional differences is obtained by taking the average of the directional differences for each pixel of the same character.

[0032] The ratio of the mean directional difference to the maximum directional difference is used as the intensity of the directional change.

[0033] Arrange the directional mutation intensity according to the corresponding text pixels, and set the directional mutation intensity of other background pixels to 0 to obtain the directional mutation intensity distribution image.

[0034] Furthermore, the formula for calculating the directional difference is:

[0035] ,

[0036] Where, △d i,k Let d be the direction difference between the i-th character pixel and the k-th character pixel in the neighborhood. max,i Let d be the dominant direction index of the i-th text pixel. max,k Let |i| be the dominant direction index of the k-th text pixel within the neighborhood of the i-th text pixel as the center, |i| be the absolute value, and min is the minimum of the two values.

[0037] Furthermore, the process of extracting stroke feature maps and transition feature maps includes:

[0038] The first image feature self-enhancement unit is used to perform a self-enhancement operation on the orientation dominance distribution image to obtain an orientation dominance enhanced feature map.

[0039] The second image feature self-enhancement unit is used to perform self-enhancement operation on the character image to obtain the character enhanced feature map;

[0040] A third image feature self-enhancement unit is used to perform a self-enhancement operation on the directional mutation intensity distribution image to obtain a directional mutation intensity enhanced feature map.

[0041] The stroke feature map is obtained by element-wise multiplication of the direction dominance enhancement feature map and the character enhancement feature map;

[0042] The transition feature map is obtained by element-wise multiplying the directional change intensity enhancement feature map with the word enhancement feature map.

[0043] Furthermore, the first image feature self-enhancing unit, the second image feature self-enhancing unit, and the third image feature self-enhancing unit have the same structure, each including: an upsampling layer, a first convolutional layer, a first Sigmoid layer, and a multiplier M3;

[0044] The input of the upsampling layer serves as the input of three image feature self-enhancement units, and its output is connected to the input of the first convolutional layer.

[0045] The output of the first convolutional layer is connected to the input of the first sigmoid layer and the first input of the multiplier M3, respectively.

[0046] The output of the first Sigmoid layer is connected to the second input of multiplier M3;

[0047] The output of multiplier M3 serves as the output of three image feature self-enhancement units.

[0048] Furthermore, the process of obtaining the identification results includes:

[0049] The first multi-level feature extraction unit is used to perform multi-level feature extraction on the stroke feature map to obtain multi-level stroke features.

[0050] The second multi-level feature extraction unit is used to perform multi-level feature extraction on the transition feature map to obtain the multi-level transition features.

[0051] The stroke multi-level features and the transition multi-level features are added element by element to obtain the fused features;

[0052] A fully connected layer is used to classify and identify the fused features to obtain the identification results.

[0053] Furthermore, the first multi-level feature extraction unit and the second multi-level feature extraction unit have the same structure, both including: a first feature extraction module, a second feature extraction module, a third feature extraction module, and a Concat layer;

[0054] The input of the first feature extraction module serves as the input of two multi-level feature extraction units, and its output is connected to the input of the second feature extraction module and the input of the Concat layer, respectively.

[0055] The output of the second feature extraction module is connected to the input of the third feature extraction module and the input of the Concat layer, respectively.

[0056] The output of the third feature extraction module is connected to the input of the Concat layer;

[0057] The output of the Concat layer serves as the output of two multi-level feature extraction units.

[0058] The beneficial effects of this invention are as follows:

[0059] 1. This invention acquires handwritten text images and segments individual characters. Based on the gradient direction of each character pixel, it obtains the directional intensity of eight directional intervals for each character pixel. The ratio of the maximum directional intensity to the total directional intensity is taken as the directional dominance, which is used to measure the effectiveness of the stroke. This facilitates the extraction of stroke feature maps by combining the character image and highlighting stroke features. At the same time, it calculates the directional change intensity for each character pixel to highlight key structures such as turns and intersections, which facilitates the extraction of turn feature maps by combining the character image. Then, it performs multi-level feature extraction and fusion on the stroke feature maps and turn feature maps. By integrating from local structure to global structure, it can preserve the details of stroke turns and capture the overall outline of the character, thereby improving recognition accuracy.

[0060] 2. This invention calculates the directional dominance and directional change intensity based on the gradient direction of pixels, highlighting key structures such as the main direction of strokes and turning intersections. Then, it performs feature enhancement and multi-level fusion on the character image, the directional dominance distribution image, and the directional change intensity distribution image to accurately capture the stroke and turning details of handwritten characters and improve recognition accuracy. Attached Figure Description

[0061] Figure 1 A flowchart of an optical character recognition method for auxiliary teaching;

[0062] Figure 2 This is a schematic diagram of the overall structure of the optical character recognition model of the present invention;

[0063] Figure 3 This is a schematic diagram of the structure of the first image feature self-enhancing unit, the second image feature self-enhancing unit, and the third image feature self-enhancing unit;

[0064] Figure 4 This is a schematic diagram of the structure of the first multi-level feature extraction unit and the second multi-level feature extraction unit;

[0065] Figure 5 This is a schematic diagram of the structure of the first feature extraction module, the second feature extraction module, and the third feature extraction module. Detailed Implementation

[0066] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0067] like Figure 1 As shown, an optical character recognition method for assisting teaching includes the following steps:

[0068] Collect images of handwritten text and segment individual characters from the images to obtain character images;

[0069] Based on the gradient direction of each character pixel in the character image, the directional intensity of 8 directional intervals is obtained;

[0070] The ratio of the maximum directional intensity to the total directional intensity of the same character pixel is used as the directional dominance, and the directional dominance distribution image is obtained.

[0071] Calculate the intensity of directional abrupt changes for each character pixel in the character image to obtain an image of the distribution of directional abrupt changes intensity.

[0072] Feature enhancement was performed on the character image, the orientation dominance distribution image, and the orientation change intensity distribution image, and stroke feature maps and transition feature maps were extracted respectively.

[0073] Multi-level feature extraction and fusion were performed on the stroke feature map and the transition feature map respectively, and a fully connected layer was used for classification and recognition to obtain the recognition results.

[0074] In this embodiment, a digital camera, scanner, or mobile device camera is used to acquire images of students' handwritten text, and individual characters are cut out from the handwritten text images to obtain character images.

[0075] The pixels in the character image are divided into two categories: background pixels and text pixels. The similarity between the pixel value of the character image and the pixel value of the stored text pixels is calculated. When the similarity is higher than the similarity threshold, the corresponding pixel is classified as a text pixel, and the other pixels are classified as background pixels.

[0076] In this embodiment, the process of obtaining the directional intensity of the eight directional intervals includes:

[0077] The horizontal gradient component of each character pixel in the character image is calculated using the horizontal gradient operator in the Prewitt operator.

[0078] The vertical gradient component of each character pixel in the character image is calculated using the vertical gradient operator in the Prewitt operator.

[0079] The gradient direction is obtained from the horizontal and vertical gradient components.

[0080] Extending the gradient direction to the range of [0°, 360°] yields the extended direction;

[0081] Divide [0°, 360°] into 8 directional intervals;

[0082] Using each character pixel as the center, the directional intensity is calculated based on the expansion direction of all character pixels within the same directional interval in the neighborhood of the center.

[0083] The formula for the gradient direction is: α = arctan(G y / G x ) or α = atan2(G y G x ), where α is the gradient direction, arctan is the arctangent function, and G y G represents the vertical gradient component. x is the horizontal gradient component, and atan2 is the azimuth function.

[0084] Extending the gradient direction to the range of [0°, 360°] is an existing technique. For example, the output range of atan2(Gy, Gx) is [-180°, 180°] (in degrees). The rule for mapping to [0°, 360°] is very simple: if the original angle α ≥ 0°, it remains unchanged (i.e., 0° ≤ α ≤ 180°); if the original angle α < 0°, it is increased by 360° (i.e., α + 360°, and the result falls in (180°, 360°)).

[0085] The eight directional intervals include: [0°, 45°), [45°, 90°), [90°, 135°), [135°, 180°), [180°, 225°), [225°, 270°), [270°, 315°), and [315°, 360°).

[0086] In this embodiment, the formula for calculating directional intensity is:

[0087] ,

[0088] Among them, E θ,i,d Let θ be the directional intensity of the d-th direction interval of the i-th text pixel. i,jLet || represent the expansion direction of the j-th character pixel within the central neighborhood when the i-th character pixel is the center, || represents the absolute value, d is the index of the direction interval, and d is a positive integer ranging from 0 to 7, N is the number of character pixels within the central neighborhood, i and j are positive integers, and cos is the cosine function.

[0089] When calculating directional intensity, the number N of text pixels within the central neighborhood includes the central text pixel.

[0090] This invention uses each character pixel as the center, statistically analyzes the expansion directions of the same directional interval within its neighborhood, and calculates the sum of the absolute values ​​of the angle differences using a cosine function. The expansion directions within a local region are then used to measure the directional strength of the central pixel in eight directional regions. By considering the directional information of multiple pixels within the central neighborhood, rather than a single pixel, the impact of noise and other interference factors on the directional strength calculation can be effectively reduced.

[0091] Dividing 0°–360° into eight directional intervals, it covers all possible stroke directions of handwritten characters (such as horizontal, vertical, left-falling, right-falling strokes and their various slanted variations), and can comprehensively capture the directional characteristics of handwritten characters of different writing styles and fonts, thus improving adaptability to diverse handwritten characters.

[0092] In this embodiment, the process of obtaining the orientation dominance distribution image includes:

[0093] The total directional intensity is obtained by summing the directional intensities of the eight directional intervals for each character pixel.

[0094] Select the maximum directional intensity from the directional intensities of the eight directional intervals for each text pixel;

[0095] The ratio of maximum directional intensity to total directional intensity is used as the directional dominance of the corresponding text pixel.

[0096] Arrange the directional dominance values ​​according to the corresponding text pixels, and set the directional dominance values ​​of other background pixels to 0 to obtain the directional dominance distribution image.

[0097] Handwritten text strokes exhibit significant "directional consistency" in their core directions (e.g., horizontal strokes are horizontal, vertical strokes are vertical), while background or noise pixels typically have chaotic orientations. The total directional intensity is the sum of eight directions, reflecting the overall directional intensity of the pixel; the maximum directional intensity is the most significant directional component. The ratio of these two (directional dominance) essentially represents the "proportion of the dominant direction in the total intensity."

[0098] The direction of the main pixels of the stroke is highly concentrated (e.g., the horizontal intensity of the middle pixel of a horizontal stroke can account for more than 80%), and the direction dominance is close to 1; while the direction of the edges or turning points of the stroke is dispersed (e.g., the end of a horizontal stroke may contain a diagonal component), and the dominance is slightly lower but still higher than that of the background; noise pixels and smeared parts have a dominance close to 0 due to chaotic direction. The greater the direction dominance, the greater the probability that the pixel is a pixel of the stroke.

[0099] In this embodiment, the process of obtaining the intensity distribution image of directional mutation includes:

[0100] For each character pixel in the character image, the direction interval corresponding to the maximum directional intensity is taken as the dominant direction;

[0101] Calculate the directional difference between each text pixel and its neighboring text pixels in the dominant direction;

[0102] The mean of the directional differences is obtained by taking the average of the directional differences for each pixel of the same character.

[0103] The ratio of the mean directional difference to the maximum directional difference is used as the intensity of the directional change.

[0104] Arrange the directional mutation intensity according to the corresponding text pixels, and set the directional mutation intensity of other background pixels to 0 to obtain the directional mutation intensity distribution image.

[0105] This invention calculates the neighborhood direction difference based on the dominant direction, enabling targeted capture of changes in stroke direction. For example, at the corner of a "horizontal-turning" stroke, the dominant direction difference between the center pixel and its neighboring pixels increases significantly. By quantifying the ratio of the mean direction difference to the maximum direction difference, a high-value area can be formed in the direction change intensity distribution image, clearly marking the turning point and solving the problem of blurred capture of such key structures by traditional methods.

[0106] In this embodiment, the maximum directional difference is set to 180°.

[0107] In this embodiment, the formula for calculating the directional difference is:

[0108] ,

[0109] Where, △d i,k Let d be the direction difference between the i-th character pixel and the k-th character pixel in the neighborhood. max,i Let d be the dominant direction index of the i-th text pixel. max,k Let |i| be the dominant direction index of the k-th text pixel within the neighborhood of the i-th text pixel as the center, |i| be the absolute value, and min is the minimum of the two values.

[0110] The dominant direction index is the index of the direction interval corresponding to the maximum directional intensity.

[0111] The angle is periodic (0°~360° cycle). Directly calculating the difference between two angles will lead to the ambiguity of "reverse equivalence". The formula ensures that the direction difference is always the minimum value in the range of 0°~180° by forcibly taking the "minimum possible angle difference", thus avoiding the error calculation caused by the periodicity of the angle.

[0112] In this embodiment, another method for calculating the intensity of directional abrupt change for each character pixel in the character image includes: calculating the directional difference between each character pixel and its neighboring character pixels in the expansion direction: Δd i,k = min(|ε max,i −ε max,k |, 360°−|ε max,i −ε max,k |); The average of the directional differences for each pixel of the same text is obtained; the ratio of the average directional difference to the maximum directional difference is used as the intensity of the directional change. Therefore, the intensity of the directional change can be calculated using the expansion direction of the text pixel. △d i,k ε is the directional difference between the i-th character pixel and the k-th character pixel in the neighborhood. max,i ε represents the expansion direction of the i-th text pixel. max,k The direction of expansion of the k-th character pixel within the neighborhood when the i-th character pixel is the center.

[0113] In this embodiment, the process of extracting stroke feature maps and transition feature maps includes:

[0114] The first image feature self-enhancement unit is used to perform a self-enhancement operation on the orientation dominance distribution image to obtain an orientation dominance enhanced feature map.

[0115] The second image feature self-enhancement unit is used to perform self-enhancement operation on the character image to obtain the character enhanced feature map;

[0116] A third image feature self-enhancement unit is used to perform a self-enhancement operation on the directional mutation intensity distribution image to obtain a directional mutation intensity enhanced feature map.

[0117] The stroke feature map is obtained by element-wise multiplication of the direction dominance enhancement feature map and the character enhancement feature map;

[0118] The transition feature map is obtained by element-wise multiplying the directional change intensity enhancement feature map with the word enhancement feature map.

[0119] This invention processes the directional dominance distribution image, character image, and directional change intensity distribution image respectively through three image feature self-enhancement units, highlighting the features of key areas. The directional dominance enhancement feature map is multiplied element-wise with the character enhancement feature map to retain the effective text area and highlight the overall stroke features. The directional change intensity enhancement feature map is multiplied element-wise with the character enhancement feature map to highlight the real turning areas within the text structure.

[0120] like Figure 2 As shown, the optical character recognition model of the present invention includes: a first image feature self-enhancing unit, a second image feature self-enhancing unit, a third image feature self-enhancing unit, a multiplier M1, a multiplier M2, a first multi-level feature extraction unit, a second multi-level feature extraction unit, an adder A1, and a fully connected layer.

[0121] The optical character recognition model of the present invention is used after training. During training, character images are collected, and the orientation dominance distribution image and orientation change intensity distribution image are extracted. The character image, the corresponding orientation dominance distribution image, and the corresponding orientation change intensity distribution image are used as samples and labeled. The model is trained using optimization algorithms (such as stochastic gradient descent SGD, Adam, etc.).

[0122] like Figure 3 As shown, the first image feature self-enhancing unit, the second image feature self-enhancing unit, and the third image feature self-enhancing unit have the same structure, all including: an upsampling layer, a first convolutional layer, a first Sigmoid layer, and a multiplier M3;

[0123] The input of the upsampling layer serves as the input of three image feature self-enhancement units, and its output is connected to the input of the first convolutional layer.

[0124] The output of the first convolutional layer is connected to the input of the first sigmoid layer and the first input of the multiplier M3, respectively.

[0125] The output of the first Sigmoid layer is connected to the second input of multiplier M3;

[0126] The output of multiplier M3 serves as the output of three image feature self-enhancement units.

[0127] In this embodiment, the kernel size of the first convolutional layer is set to 1×1. This invention improves the feature map resolution through an upsampling layer, extracts features through the first convolutional layer, generates attention weights through the first sigmoid layer, and multiplies them element-wise with the original features (M3 operation). This adaptively highlights key features, suppresses irrelevant information, and enhances the relevance and effectiveness of feature representation.

[0128] In this embodiment, the process of obtaining the recognition result includes:

[0129] The first multi-level feature extraction unit is used to perform multi-level feature extraction on the stroke feature map to obtain multi-level stroke features.

[0130] The second multi-level feature extraction unit is used to perform multi-level feature extraction on the transition feature map to obtain the multi-level transition features.

[0131] The stroke multi-level features and the transition multi-level features are added element by element to obtain the fused features;

[0132] A fully connected layer is used to classify and identify the fused features to obtain the identification results.

[0133] This invention fully extracts feature information from different levels of stroke feature maps and transition feature maps through first and second multi-level feature extraction units. Then, the two are added element by element to obtain fused features, which can integrate the complementary features of the overall shape of the strokes and the transition details of the characters. Finally, the fully connected layer is used for classification and recognition, which not only enriches the feature dimensions but also enhances the ability to represent the structure of the characters, thereby improving the accuracy and robustness of the recognition results.

[0134] like Figure 4 As shown, the first multi-level feature extraction unit and the second multi-level feature extraction unit have the same structure, both including: a first feature extraction module, a second feature extraction module, a third feature extraction module, and a Concat layer;

[0135] The input of the first feature extraction module serves as the input of two multi-level feature extraction units, and its output is connected to the input of the second feature extraction module and the input of the Concat layer, respectively.

[0136] The output of the second feature extraction module is connected to the input of the third feature extraction module and the input of the Concat layer, respectively.

[0137] The output of the third feature extraction module is connected to the input of the Concat layer;

[0138] The output of the Concat layer serves as the output of two multi-level feature extraction units.

[0139] This invention extracts input features sequentially and across levels through first, second, and third feature extraction modules. Then, it uses a Concat layer to splice and fuse the features extracted at different stages. This fully integrates feature information from shallow to deep levels, preserving the details of shallow features (such as edges and local structures) while incorporating the abstract semantics of deep features (such as overall patterns and category tendencies). This greatly enriches the expressive dimensions and representational capabilities of the features, providing more comprehensive and discriminative feature support for subsequent tasks (such as classification and recognition), thereby improving the accuracy and robustness of the tasks.

[0140] like Figure 5 As shown, the first feature extraction module, the second feature extraction module, and the third feature extraction module have the same structure, each including: a max pooling layer, an average pooling layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, and an adder A2;

[0141] The input of the max pooling layer is connected to the input of the average pooling layer, and serves as the input of the three feature extraction modules;

[0142] The input of the second convolutional layer is connected to the output of the max pooling layer, and its output is connected to the input of the third convolutional layer.

[0143] The input of the fourth convolutional layer is connected to the output of the average pooling layer, and its output is connected to the input of the fifth convolutional layer.

[0144] The input of adder A2 is connected to the output of the third convolutional layer and the output of the fifth convolutional layer, respectively, and its output serves as the output of the three feature extraction modules.

[0145] In this embodiment, the kernel size of the second, third, fourth, and fifth convolutional layers is 3×3.

[0146] This invention captures the local maximum response and global average information of features through max pooling and average pooling layers, respectively. Then, features of different dimensions are further extracted through multiple convolutions, and finally fused by adder A2.

[0147] This invention acquires handwritten text images and segments individual characters. Based on the gradient direction of each character pixel, it obtains the directional intensity of eight directional intervals for each character pixel. The ratio of the maximum directional intensity to the total directional intensity is taken as the directional dominance, which is used to measure the effectiveness of the stroke. This facilitates the extraction of stroke feature maps by combining the character image and highlighting stroke features. At the same time, it calculates the directional change intensity for each character pixel, highlighting key structures such as turns and intersections, which facilitates the extraction of turn feature maps by combining the character image. Then, it performs multi-level feature extraction and fusion on the stroke feature maps and turn feature maps. By integrating from local structure to global structure, it can preserve the details of stroke turns and capture the overall outline of the character, thereby improving recognition accuracy.

[0148] This invention calculates the directional dominance and directional change intensity based on the gradient direction of pixels, highlighting key structures such as the main direction of strokes and turning intersections. Then, it performs feature enhancement and multi-level fusion on the character image, the directional dominance distribution image, and the directional change intensity distribution image to accurately capture the stroke and turning details of handwritten characters and improve recognition accuracy.

[0149] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optical character recognition to assist in teaching, characterized in that, Includes the following steps: Collect images of handwritten text and segment individual characters from the images to obtain character images; Based on the gradient direction of each character pixel in the character image, the directional intensity of 8 directional intervals is obtained; The ratio of the maximum directional intensity to the total directional intensity of the same character pixel is used as the directional dominance, and the directional dominance distribution image is obtained. The horizontal gradient component of each character pixel in the character image is calculated using the horizontal gradient operator in the Prewitt operator. The vertical gradient component of each character pixel in the character image is calculated using the vertical gradient operator in the Prewitt operator. The gradient direction is obtained from the horizontal and vertical gradient components. Extending the gradient direction to the range of [0°, 360°] yields the extended direction; Divide [0°, 360°] into 8 directional intervals; Taking each text pixel as the center, the directional intensity is calculated based on the expansion direction of all text pixels in the same directional interval within the center's neighborhood. The formula for calculating directional strength is: ， Among them, E θ,i,d Let θ be the directional intensity of the d-th direction interval of the i-th text pixel. i,j Let || represent the expansion direction of the j-th character pixel within the central neighborhood when the i-th character pixel is the center, || represents the absolute value, d is the index of the direction interval, and the value of d is a positive integer from 0 to 7, N is the number of character pixels within the central neighborhood, i and j are positive integers, and cos is the cosine function; Calculate the intensity of directional abrupt changes for each character pixel in the character image to obtain an image of the distribution of directional abrupt changes intensity. For each character pixel in the character image, the direction interval corresponding to the maximum directional intensity is taken as the dominant direction; Calculate the directional difference between each text pixel and its neighboring text pixels in the dominant direction; The mean of the directional differences is obtained by taking the average of the directional differences for each pixel of the same character. The ratio of the mean directional difference to the maximum directional difference is used as the intensity of the directional change. Feature enhancement was performed on the character image, the orientation dominance distribution image, and the orientation change intensity distribution image, and stroke feature maps and transition feature maps were extracted respectively. The first image feature self-enhancement unit is used to perform a self-enhancement operation on the orientation dominance distribution image to obtain an orientation dominance enhanced feature map. The second image feature self-enhancement unit is used to perform self-enhancement operation on the character image to obtain the character enhanced feature map; A third image feature self-enhancement unit is used to perform a self-enhancement operation on the directional mutation intensity distribution image to obtain a directional mutation intensity enhanced feature map. The stroke feature map is obtained by element-wise multiplication of the direction dominance enhancement feature map and the character enhancement feature map; The transition feature map is obtained by element-wise multiplying the directional change intensity enhancement feature map with the word enhancement feature map; The first image feature self-enhancing unit, the second image feature self-enhancing unit, and the third image feature self-enhancing unit have the same structure, all including: an upsampling layer, a first convolutional layer, a first Sigmoid layer, and a multiplier M3; The input of the upsampling layer serves as the input of three image feature self-enhancement units, and its output is connected to the input of the first convolutional layer. The output of the first convolutional layer is connected to the input of the first sigmoid layer and the first input of the multiplier M3, respectively. The output of the first Sigmoid layer is connected to the second input of multiplier M3; The output of multiplier M3 serves as the output of three image feature self-enhancement units; Multi-level feature extraction and fusion were performed on the stroke feature map and the transition feature map respectively, and a fully connected layer was used for classification and recognition to obtain the recognition results.

2. The optical character recognition method for auxiliary teaching according to claim 1, characterized in that, The process of obtaining the orientation dominance distribution image includes: The total directional intensity is obtained by summing the directional intensities of the eight directional intervals for each character pixel. Select the maximum directional intensity from the directional intensities of the eight directional intervals for each text pixel; The ratio of maximum directional intensity to total directional intensity is used as the directional dominance of the corresponding text pixel. Arrange the directional dominance values ​​according to the corresponding text pixels, and set the directional dominance values ​​of other background pixels to 0 to obtain the directional dominance distribution image.

3. The optical character recognition method for auxiliary teaching according to claim 1, characterized in that, The process of obtaining the intensity distribution image of directional mutations includes: Arrange the directional mutation intensity according to the corresponding text pixels, and set the directional mutation intensity of other background pixels to 0 to obtain the directional mutation intensity distribution image.

4. The optical character recognition method for auxiliary teaching according to claim 3, characterized in that, The formula for calculating the direction difference is: ， Where, △d i,k Let d be the direction difference between the i-th character pixel and the k-th character pixel in the neighborhood. max,i Let d be the dominant direction index of the i-th text pixel. max,k Let |i| be the dominant direction index of the k-th text pixel within the neighborhood of the i-th text pixel as the center, |i| be the absolute value, and min is the minimum of the two values.

5. The optical character recognition method for auxiliary teaching according to claim 1, characterized in that, The process of obtaining the recognition results includes: The first multi-level feature extraction unit is used to perform multi-level feature extraction on the stroke feature map to obtain multi-level stroke features. The second multi-level feature extraction unit is used to perform multi-level feature extraction on the transition feature map to obtain the multi-level transition features. The stroke multi-level features and the transition multi-level features are added element by element to obtain the fused features; A fully connected layer is used to classify and identify the fused features to obtain the identification results.

6. The optical character recognition method for auxiliary teaching according to claim 5, characterized in that, The first multi-level feature extraction unit and the second multi-level feature extraction unit have the same structure, both including: a first feature extraction module, a second feature extraction module, a third feature extraction module, and a Concat layer; The input of the first feature extraction module serves as the input of two multi-level feature extraction units, and its output is connected to the input of the second feature extraction module and the input of the Concat layer, respectively. The output of the second feature extraction module is connected to the input of the third feature extraction module and the input of the Concat layer, respectively. The output of the third feature extraction module is connected to the input of the Concat layer; The output of the Concat layer serves as the output of two multi-level feature extraction units.

Images