Character recognition system and method and storage medium

Abstract

The invention relates to a character recognition system and method and a storage medium, and relates to the technical field of character recognition, and the system comprises an input module which is used for reading an original image; the preprocessing module is used for preprocessing the original image; the character detection module is used for extracting a character region image in the original image; the character segmentation module is used for segmenting the character region image to obtain a single character image; the character recognition module is used for extracting character features of the single-character image and inputting the character features into a character recognition model to obtain a character recognition result, and the character recognition model is used for determining a character probability value matched with the character features and determining the character recognition result; the decryption module is used for decrypting the encrypted characters; the output module is used for outputting the decrypted text; the control module is used for calling all the modules in sequence. According to the scheme, the time and the error rate of manual operation can be reduced, and the processing efficiency is remarkably improved.

Description

Technical Field

[0001] This disclosure relates to the field of character recognition technology, specifically to a character recognition system, method, and storage medium. Background Technology

[0002] Character recognition technology is widely used in various scenarios, such as document digitization, automatic data entry, license plate recognition, and identity verification.

[0003] Existing character recognition methods mainly rely on manual character recognition. Although manual character recognition can provide higher accuracy, especially when dealing with complex or ambiguous characters, the speed of manual character recognition is much lower than that of automated methods, especially when dealing with a large number of characters or complex documents, thus reducing the efficiency of character recognition. Summary of the Invention

[0004] In view of this, this disclosure provides a character recognition system, method and storage medium, the main purpose of which is to solve the problem that the speed of manual character recognition is much lower than that of automated methods, especially when processing a large number of characters or complex documents, thereby reducing the efficiency of character recognition.

[0005] According to the present invention, a character recognition system includes: an input module, a preprocessing module, a character detection module, a character segmentation module, a character recognition module, a decryption module, an output module, and a control module, wherein the input module, the preprocessing module, the character detection module, the character segmentation module, the character recognition module, the decryption module, and the output module are all connected to the control module;

[0006] The input module is used to read the original image, which is an irregularly shaped screenshot.

[0007] The preprocessing module is used to perform preprocessing operations on the original image;

[0008] The character detection module is used to extract character region images from the original image after preprocessing.

[0009] The character segmentation module is used to segment the character region image based on morphological operations to obtain a single character image;

[0010] The character recognition module is used to extract character features from the single character image, input the character features into the character recognition model for character recognition processing to obtain character recognition results, the character recognition model is used to determine the character probability value that matches the character features, and determine the character corresponding to the highest character probability value as the character recognition result, the character recognition result includes encrypted characters;

[0011] The decryption module is used to decrypt the encrypted characters;

[0012] The output module is used to output the decrypted text;

[0013] The control module is used to call each module in sequence to realize the automated process of character recognition.

[0014] Optionally, the preprocessing includes image correction processing;

[0015] The preprocessing module is specifically used to: determine the corner coordinates of the original image and the vertex coordinates of the target image; determine the geometric transformation matrix based on the corner coordinates and the vertex coordinates; and use the geometric transformation matrix to geometrically transform the original image into the target image, wherein the target image is a regular screenshot image obtained by geometric transformation of the original image, and the geometric transformation matrix is ​​used to describe the coordinate mapping relationship from the original image to the target image.

[0016] Optionally, the preprocessing includes image denoising.

[0017] The preprocessing module is specifically used for: identifying Gaussian noise and / or salt-and-pepper noise in the original image; performing denoising processing on the original image using nonlocal mean filtering to obtain a first filtered image; performing smoothing processing on the original image using bilateral filtering to obtain a second filtered image; performing weighted fusion operation on the first filtered image and the second filtered image to suppress the Gaussian noise; and / or, performing noise point probability prediction on each pixel in the original image to obtain the probability value of each pixel being a salt-and-pepper noise point, and performing median filtering processing on pixels with probability values ​​greater than a preset probability threshold to suppress the salt-and-pepper noise.

[0018] Optionally, the preprocessing includes binarization.

[0019] The preprocessing module is specifically used to: traverse each pixel in the original image and compare the pixel value of each pixel with a preset pixel threshold; set pixels with pixel values ​​greater than the preset pixel threshold to white; and set pixels with pixel values ​​less than or equal to the preset pixel threshold to black.

[0020] Optionally, the character detection module is specifically used for: performing edge detection on the original image to obtain an edge image; performing contour detection on the edge image to obtain all contours; filtering character contours among all contours based on character contour features; and extracting the corresponding character region image from the original image based on the character contours.

[0021] Optionally, the morphological operations include at least one of dilation, erosion, opening, and closing operations;

[0022] The character segmentation module is specifically used for: performing a dilation operation on the character region image to connect adjacent character parts; performing an erosion operation on the character region image to separate adhered character parts; performing an opening operation on the character region image to remove noise and small connected parts; performing a closing operation on the character region image to fill internal holes in the characters; performing connected component analysis on the character region image after morphological operations to obtain connected component analysis results; and extracting each connected component from the character region image to form the single character image based on the connected component analysis results.

[0023] Optionally, the system further includes: a model training module; the model training module is used to acquire historical single-character images and historical character recognition results corresponding to the historical single-character images; extract historical character features from the historical single-character images, use the historical character features as input features of the character recognition model, use the historical character recognition results as training labels of the character recognition model, iteratively train the character recognition model until the loss function value of the character recognition model is less than a preset loss function threshold, and determine that the character recognition model training is complete.

[0024] Optionally, the decryption module is specifically used to: decrypt the encrypted characters using a Caesar cipher or a substitution cipher;

[0025] The system also includes: a reminder module;

[0026] The reminder module is used to record relevant information about the decryption failure and send a decryption failure reminder to the user if the decryption fails.

[0027] Optionally, each module can be encapsulated as an independent function or class.

[0028] According to a second aspect of this disclosure, a character recognition method is provided, the method comprising:

[0029] Read the original image, which is an irregularly shaped screenshot;

[0030] The original image is preprocessed, and the character region image is extracted from the original image after the preprocessing operation.

[0031] The character region image is segmented based on morphological operations to obtain a single character image;

[0032] The character features of the single-character image are extracted, and the character features are input into a character recognition model for character recognition processing to obtain a character recognition result. The character recognition model is used to determine the probability value of the character that matches the character features. The character corresponding to the highest probability value is determined as the character recognition result. The character recognition result includes encrypted characters.

[0033] The encrypted characters are decrypted, and the decrypted text is output.

[0034] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method of the second aspect described above.

[0035] By employing the above technical solutions, this disclosure provides a character recognition system, method, and storage medium. Compared with existing technologies, this disclosure allows for the following: an input module reads an original image, which is an irregularly shaped screenshot; a preprocessing module performs preprocessing operations on the original image; a character detection module extracts character region images from the preprocessed original image; a character segmentation module segments the character region images based on morphological operations to obtain single-character images; a character recognition module extracts character features from the single-character images, inputs these features into a character recognition model for character recognition processing to obtain character recognition results, where the character recognition model determines the probability value of characters matching the character features, and the character corresponding to the highest probability value is identified as the character recognition result, which includes encrypted characters; a decryption module decrypts the encrypted characters; an output module outputs the decrypted text; and a control module sequentially calls each module to automate the character recognition process.

[0036] The solution disclosed herein automatically schedules the entire process, from original image input to decrypted text output, without manual intervention. Specifically, when processing a large number of irregularly shaped screenshots, the system can automatically complete image reading, preprocessing, character detection, segmentation, recognition, decryption, and output, reducing manual operation time and error rate, and significantly improving processing efficiency.

[0037] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0039] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A schematic diagram of the structure of a character recognition system provided in an embodiment of this disclosure is shown;

[0041] Figure 2 A schematic diagram of another character recognition system provided in an embodiment of this disclosure is shown;

[0042] Figure 3 A flowchart illustrating a character recognition method provided in an embodiment of this disclosure is shown.

[0043] Figure 1 middle:

[0044] 1-Input module;

[0045] 2-Preprocessing module;

[0046] 3-Character detection module;

[0047] 4-Character splitting module;

[0048] 5-Character recognition module;

[0049] 6-Decryption module;

[0050] 7-Output Module;

[0051] 8-Control module;

[0052] Figure 2 middle:

[0053] 9-Model Training Module;

[0054] 10-Reminder Module. Detailed Implementation

[0055] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0056] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0057] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0058] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0059] To address the issue that manual character recognition is significantly slower than automated methods, especially when processing large amounts of characters or complex documents, thus reducing character recognition efficiency, this disclosure provides a character recognition system, method, and storage medium.

[0060] like Figure 1 As shown, embodiments of this disclosure provide a character recognition system, which includes: an input module 1, a preprocessing module 2, a character detection module 3, a character segmentation module 4, a character recognition module 5, a decryption module 6, an output module 7, and a control module 8;

[0061] The input module 1, preprocessing module 2, character detection module 3, character segmentation module 4, character recognition module 5, decryption module 6, and output module 7 are all connected to the control module 8.

[0062] In this embodiment of the disclosure, the input module 1 can be used to receive externally read original images, which may include, but are not limited to, images obtained by scanners, digital cameras, smartphones, screenshot tools, networks, etc.; the original image is an irregular screenshot image, wherein the irregular screenshot image may be a low-quality screenshot image captured by the user through mobile devices such as mobile phones and computers, which has different sizes, tilted angles, blur, noise and / or background interference.

[0063] In this embodiment of the disclosure, the preprocessing module 2 can be used to perform preprocessing operations on the received raw image. Preprocessing operations may include, but are not limited to, image correction, background removal, resizing, grayscale conversion, binarization, and image denoising.

[0064] Among them, image correction processing can be used to correct irregular screenshots into regular rectangular shapes; background removal can be used to remove the background of an image while retaining foreground characters; size adjustment can be used to adjust the image size to meet the requirements of subsequent processing modules; grayscale conversion can be used to convert color images to grayscale images; binarization processing can be used to convert grayscale images to binary images; and image denoising processing can be used to remove noise points in an image.

[0065] In the embodiments of this disclosure, image correction processing can be used to determine the corner coordinates of the original image and the vertex coordinates of the target image, determine the geometric transformation matrix based on the corner coordinates and vertex coordinates, and use the geometric transformation matrix to geometrically transform the original image into the target image. The target image is a regular screenshot image obtained by geometric transformation of the original image, and the geometric transformation matrix is ​​used to describe the coordinate mapping relationship from the original image to the target image.

[0066] Specifically, the coordinates of the four corner points of the original image can be determined using image processing algorithms. In document scanning scenarios, the detected corner points are typically the four corners of the document. By setting appropriate thresholds and filtering rules, these four key corner points can be accurately extracted from numerous detected corner points. Corner point coordinates can be obtained using various corner detection algorithms, such as Harris corner detection, Shi-Tomasi corner detection, and SIFT (Scale Invariant Feature Transform) algorithm.

[0067] The coordinates of the four vertices of the target image can be determined based on the corner coordinates of the original image. Specifically, the vertex coordinates of the target image can be defined according to the shape and size requirements of the target image. For example, if the goal is to correct the original image into a rectangle, and the width of the target rectangle is known to be `target_width` and the height to be `target_height`, then the coordinates of the four vertices of the target image can be defined as (0,0), (target_width,0), (target_width,target_height), and (0,target_height).

[0068] The appropriate geometric transformation matrix type can be selected based on the degree of distortion in the original image and the geometric transformation requirements of the target image. Geometric transformation matrix types can include affine matrices and homography matrices. If the original image only undergoes linear transformations such as translation, rotation, scaling, and shearing, an affine matrix is ​​used for correction; this affine matrix is ​​a 2×3 matrix. If the original image exhibits perspective distortion (e.g., the camera and document plane are not parallel when photographing a document), a homography matrix can be used for correction; this homography matrix is ​​a 3×3 matrix.

[0069] The geometric transformation matrix can be calculated using the determined corner coordinates of the original image and the vertex coordinates of the target image through OpenCV's affine transformation function or homography transformation function.

[0070] If using an affine matrix, the original image can be affinely transformed using OpenCV's affine transformation functions to obtain a corrected, regular image. If using a homography matrix, the original image can be homography transformed using OpenCV's homography matrix functions to obtain a corrected, regular image.

[0071] During image transformation, inaccurate corner point localization or errors in transformation matrix calculation may lead to distortion or deformation in the corrected image. The correction effect can be improved by adjusting the corner point coordinate selection rules and optimizing the matrix calculation algorithm (e.g., using the RANSAC algorithm to optimize homography matrix calculation and remove the influence of outliers).

[0072] Through the above implementation method, image correction processing can convert irregularly shaped screenshots into regularly shaped screenshots, providing more accurate input for subsequent image processing. This implementation method can be applied to various image processing systems, improving the efficiency and accuracy of image processing.

[0073] In this embodiment of the disclosure, image denoising processing can be used to identify Gaussian noise and / or salt-and-pepper noise in the original image; a first filtered image is obtained by denoising the original image using non-local means (NLM), a second filtered image is obtained by smoothing the original image using bilateral filtering, and a weighted fusion operation is performed on the first filtered image and the second filtered image to suppress Gaussian noise; and / or, noise point probability prediction is performed on each pixel in the original image to obtain the probability value of each pixel being a salt-and-pepper noise point, and median filtering is performed on pixels with probability values ​​greater than a preset probability threshold to suppress salt-and-pepper noise.

[0074] In specific application scenarios, users can use common screenshot tools (such as the built-in screenshot tool in Windows, Snipaste, WeChat screenshot, etc.) to take screenshots of target interfaces (such as web pages, software windows, document pages, etc.) and obtain the original image containing Gaussian noise and / or salt-and-pepper noise. For example, when browsing web pages and taking screenshots in low-light environments, Gaussian noise may appear in the original image due to the influence of screen display noise and image sensor noise; or when taking screenshots under unstable network transmission conditions, salt-and-pepper noise may be mixed into the original image.

[0075] Feature extraction from the original image can include calculating statistical features such as the image's gray-level histogram, gradient magnitude histogram, and local contrast. Gaussian noise can make the image's gray-level histogram appear as an approximate Gaussian distribution, and the gradient magnitude histogram is relatively smooth; salt-and-pepper noise, on the other hand, will produce obvious peaks in the gray-level histogram, and anomalies will appear in the gradient magnitude histogram corresponding to high gradient value regions.

[0076] Based on the extracted features and pre-defined judgment rules, the type of noise present in the original image can be identified. For example, if the gray-level histogram approximates a Gaussian distribution and the gradient magnitude histogram is relatively smooth, while local contrast analysis shows a certain degree of overall blurring, then Gaussian noise is identified in the image; if the gray-level histogram shows obvious peaks and the high gradient value region in the gradient magnitude histogram is abnormally prominent, then salt-and-pepper noise is identified in the image; if both features are relatively obvious, then both Gaussian noise and salt-and-pepper noise are identified in the image.

[0077] This disclosure utilizes nonlocal mean filtering to suppress Gaussian noise. Nonlocal mean filtering is based on the principle of image self-similarity. It searches for other blocks similar to the current pixel in the image and performs a weighted average of these similar blocks according to their similarity, thereby estimating the true value of the current pixel and effectively removing Gaussian noise. This method fully considers the global information of the image and can effectively preserve the texture and details of the image while removing noise.

[0078] This disclosure describes a bilateral filtering method for smoothing original images. The bilateral filtering method combines spatial proximity and pixel value similarity to calculate the filtering weights. The spatial proximity weight considers the spatial distance between pixels, while the pixel value similarity weight considers the difference in grayscale values ​​between pixels. In this way, bilateral filtering can effectively preserve edge information while smoothing the image, demonstrating good performance in removing Gaussian noise and maintaining clear image edges.

[0079] This disclosure performs a weighted fusion operation on the first filtered image obtained by nonlocal mean filtering and the second filtered image obtained by bilateral filtering, combining the advantages of the two filtering methods to further suppress Gaussian noise. The weighted fusion formula can be:

[0080] Ifused(x,y)=w1×INLM(x,y)+w2×IBilateral(x,y)

[0081] Where Ifused(x,y) represents the pixel value at pixel (x,y) of the fused image, INLM(x,y) represents the pixel value at pixel (x,y) of the first filtered image, IBilateral(x,y) represents the pixel value at pixel (x,y) of the second filtered image, and w1 and w2 represent weight coefficients, with w1+w2=1.

[0082] The values ​​of the weighting coefficients w1 and w2 can be adjusted according to the actual application scenario and the effects of the two filtering methods. For example, if nonlocal mean filtering is more effective in removing Gaussian noise, the value of w1 can be increased appropriately; if bilateral filtering is more advantageous in preserving edges, the value of w2 can be increased appropriately.

[0083] In this embodiment of the disclosure, noise point probability prediction is performed on each pixel in the original image to obtain the probability value of each pixel being a salt-and-pepper noise point. The noise point probability prediction can be implemented using a machine learning model, such as a random forest or support vector machine.

[0084] One possible approach is to construct a noise point probability prediction model based on a convolutional neural network (CNN) to predict the probability that each pixel in the original image is a salt-and-pepper noise point. This model can employ an encoder-decoder structure. The encoder consists of multiple convolutional and pooling layers to extract image features; the decoder uses deconvolutional layers and upsampling operations to restore the feature map to the original image size and outputs the noise probability for each pixel. The original image can be input into the trained noise point probability prediction model to obtain the probability value of each pixel being a salt-and-pepper noise point.

[0085] In this embodiment of the disclosure, pixels with probability values ​​greater than a preset probability threshold are subjected to median filtering to suppress salt-and-pepper noise. Median filtering is a non-linear filtering method that sorts the pixel values ​​in the neighborhood of a pixel and takes the median value as the filtering result for that pixel, thereby effectively removing salt-and-pepper noise. This method has a good suppression effect on isolated noise points while also preserving the edge information of the image relatively well.

[0086] The preset probability threshold can be used to determine whether a pixel is a salt-and-pepper noise point. A higher preset probability threshold results in a stricter judgment of noise points, which may miss some noise points; a lower preset probability threshold may misclassify some normal pixels as noise points. The preset probability threshold can be adjusted according to the noise distribution and denoising requirements in the actual application.

[0087] In this embodiment of the disclosure, if both Gaussian noise and salt-and-pepper noise exist in the original image, the fused image obtained from the Gaussian noise suppression step and the median-filtered image obtained from the salt-and-pepper noise suppression step are further fused (a suitable fusion method can be selected according to actual needs, such as simple pixel value averaging or weighted fusion based on region characteristics). If only one type of noise exists, the corresponding denoising result image is directly output.

[0088] The Matplotlib library can be used to display the original image and the denoised image side-by-side on the screen, allowing users to easily compare the effects before and after denoising. A save dialog box is also provided, allowing users to save the denoised image to a specified folder, with the save format selectable from common formats such as PNG and JPEG.

[0089] Through the above specific embodiments, the fusion filtering and denoising method for screenshot images disclosed herein can effectively suppress Gaussian noise and / or salt-and-pepper noise, thereby improving the quality of screenshot images.

[0090] In this embodiment of the disclosure, binarization processing can be used to traverse each pixel in the original image and compare the pixel value of each pixel with a preset pixel threshold; set pixels with pixel values ​​greater than the preset pixel threshold to white; and set pixels with pixel values ​​less than or equal to the preset pixel threshold to black.

[0091] Specifically, a preset pixel threshold can be determined based on image statistical information. One possible method is to perform statistical analysis on the pixel values ​​of the original image to calculate its gray-level histogram. The gray-level histogram reflects the distribution of pixels at different gray levels in the image. By analyzing the peak and valley characteristics of the histogram, an appropriate threshold can be selected. For example, for a histogram with obvious bimodal characteristics, the gray value corresponding to the valley between the two peaks is usually a good threshold choice, effectively separating the target from the background in the image.

[0092] A preset pixel threshold can be determined based on an adaptive thresholding algorithm. As one possible approach, considering that lighting conditions may vary in different regions of an image, a global threshold may not meet the needs of all regions. The adaptive thresholding algorithm calculates the threshold for each pixel based on the local features of the image. For example, using a local average or Gaussian weighted average as the local threshold can better adapt to changes in lighting and improve the binarization effect.

[0093] A preset pixel threshold can be determined manually and interactively. In some cases, users may have specific requirements for target features in an image, necessitating manual intervention in threshold selection. One possible approach is to provide an interactive interface that allows users to view the binarization effect under different thresholds in real time and adjust the threshold according to their needs until a satisfactory result is obtained.

[0094] You can use the image iterator provided by OpenCV or directly iterate through each pixel in the original image using array indexing. For grayscale images, the pixel value of each pixel is an integer between 0 and 255; for color images, they are usually converted to grayscale first and then binarized.

[0095] The pixel value of each pixel can be compared with a preset pixel threshold. If the pixel value is greater than the preset pixel threshold, the pixel is set to white (pixel value 255); if the pixel value is less than or equal to the preset pixel threshold, the pixel is set to black (pixel value 0).

[0096] The OpenCV `imshow` function can be used to display the original image and the binarized image side-by-side on the screen, allowing users to easily compare the effects before and after binarization. A save dialog box is also provided, allowing users to save the binarized image to a specified folder, with the save format selectable from common formats such as PNG and TIFF.

[0097] Through the above specific embodiments, the image pixel conversion method based on binarization processing disclosed herein can reasonably determine the preset pixel threshold according to different application scenarios and needs, effectively perform binarization processing on the original image, and improve the efficiency and accuracy of image processing.

[0098] In this embodiment of the disclosure, the character detection module 3 can be used to detect character regions in the preprocessed original image, extract character regions from the original image, and generate character region images. To improve the accuracy of character region recognition, as a possible method, edge detection can be performed on the original image to obtain an edge image; contour detection can be performed on the edge image to obtain all contours; character contours can be selected from all contours based on character contour features; and the corresponding character region images in the original image can be extracted based on the character contours.

[0099] In this embodiment of the disclosure, the Canny edge detection algorithm can be used to obtain the edge image of the image. This algorithm has good edge detection performance, can effectively suppress noise interference, and accurately detect edge information in the image. The Canny edge detection algorithm may include steps such as Gaussian filtering, calculating image gradient, non-maximum suppression, and double threshold detection.

[0100] Edge detection can be achieved using OpenCV's Canny function. First, a Gaussian filter is used to smooth the image to suppress noise. Then, the gradient magnitude and gradient direction of each pixel in the image are calculated. Next, a non-maximum suppression method is used to refine the gradient magnitude, retaining the pixels with the largest local gradient magnitude as candidate edge points. Finally, a double threshold algorithm is used to determine the final edge pixels, resulting in an edge image containing various edge information in the image.

[0101] Contour detection can be performed on edge images. The `findContours` function from the OpenCV library is used, which can find and return all contours in an image. A contour can be a curve composed of a series of continuous points, representing the boundary of an object in the image. Contour detection allows the acquisition of contour information for all objects in the image.

[0102] Based on the geometric and topological features of the characters, all detected contours are filtered to determine possible character contours. The specific filtering rules are as follows:

[0103] One possible approach is to calculate the area of ​​each contour, set an appropriate area threshold range, and retain only contours whose area falls within that threshold range. Character contours typically have a certain area size; contours that are too small or too large are likely noise or non-character objects. Area filtering can effectively remove interfering contours.

[0104] One possible approach is to calculate the aspect ratio of the bounding rectangle of each contour, set a threshold range for the aspect ratio, and retain only contours with aspect ratios within that threshold range. Characters typically have relatively regular aspect ratios; for example, Chinese characters are generally approximating squares or rectangles with aspect ratios within a certain range. Filtering by aspect ratio can exclude non-character contours with abnormal aspect ratios.

[0105] One possible approach is to calculate the ratio of the perimeter to the area of ​​each contour, i.e., the contour complexity. A threshold range for contour complexity is set, and only contours with a complexity within this range are retained. Character contours typically have relatively simple shapes, and their contour complexity falls within a certain range. Filtering by contour complexity can remove non-character contours with overly complex shapes.

[0106] For each character outline obtained from the filtering, determine its minimum bounding rectangle, take the area enclosed by the minimum bounding rectangle as the character region, and extract the image of the character region from the original image.

[0107] The extracted character region image can be further optimized. Specifically, an adaptive threshold binarization algorithm, such as the Otsu algorithm, can be used to binarize the character region image, dividing the pixels in the image into character pixels and non-character pixels, making the contrast between the characters and the background more obvious, which is convenient for subsequent character recognition.

[0108] This disclosure combines Canny edge detection and contour detection techniques to effectively identify character regions from irregularly shaped screenshots. Canny edge detection accurately detects edge information in an image, while contour detection further extracts the contours of objects. By comprehensively utilizing these two techniques, the edge and shape features of characters can be captured more comprehensively, improving the accuracy of character region recognition.

[0109] In the contour filtering process, this disclosure uses a variety of geometric and topological features for filtering, such as area, aspect ratio and contour complexity, which can effectively remove noise and non-character contours, improve the accuracy and robustness of character contour extraction, and reduce interference in subsequent character recognition.

[0110] In this embodiment of the disclosure, for characters that are attached together (such as handwritten characters or artistic fonts) or closely arranged characters (such as CAPTCHAs), a segmentation algorithm is used to separate individual characters from continuous text to avoid OCR misrecognition.

[0111] In this embodiment of the disclosure, the character segmentation module 4 can be used to segment a character region image based on morphological operations to obtain a single character image. The morphological operations include at least one of dilation, erosion, opening, and closing operations.

[0112] The dilation operation can be implemented using OpenCV's `dilate` function. It scans the image using a structuring element (kernel) and uses the maximum pixel value within the structuring element's coverage area as the output pixel value, thus expanding the character edges outward and connecting adjacent characters. The erosion operation can be implemented using OpenCV's `erode` function. It scans the image using a structuring element and uses the minimum pixel value within the structuring element's coverage area as the output pixel value, thus shrinking the character edges inward and separating connected characters. The opening operation can be implemented using OpenCV's `morphologyEx` function to remove small noise points in character regions, while the closing operation can fill small holes inside characters, making the character shape more complete and clearer.

[0113] Specifically, the character region image can be dilated to connect adjacent character parts; eroded to separate adhered character parts; opened to remove noise and small connected parts; closed to fill internal holes in the character region image; connected component analysis is performed on the morphologically operated character region image to obtain the connected component analysis results; and each connected component is extracted from the character region image to form a single character image based on the connected component analysis results.

[0114] One possible approach is to use OpenCV's `connectedComponentsWithStats` function to perform connected component analysis on the morphologically manipulated character region image, identifying all connected components in the image. This function returns not only the number of connected components but also the label, bounding box, and area of ​​each component. Character connected components can be filtered based on features such as area and aspect ratio. Components with excessively small or large areas, as well as those whose aspect ratios do not match the character characteristics, are removed. Based on the filtered valid connected component information, the individual character images corresponding to each connected component can be extracted from the original character region image using array slicing, and the extracted individual character images are saved to a list.

[0115] One possible approach is to post-process the extracted single-character images, such as resizing the image and normalizing pixel values, to facilitate subsequent character recognition operations. The OpenCV `imshow` function can be used to display the original character region image, the image after morphological operations, the connected component analysis result image, and the extracted single-character image side-by-side on the screen, allowing users to easily compare the effects before and after processing.

[0116] Through the above specific implementation methods, the character segmentation method based on morphological operations and connected component analysis disclosed herein can comprehensively consider problems such as character adhesion, noise interference, and internal holes of characters, accurately and efficiently segmenting single character images from character region images, providing high-quality input for subsequent character recognition.

[0117] In this embodiment of the disclosure, the character recognition module 5 can be used to extract character features from a single-character image and input the character features into a character recognition model for character recognition processing to obtain the character recognition result. The character recognition model can be a deep learning model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). The character recognition model can determine the probability value of the character matching the character features and determine the character corresponding to the highest probability value as the character recognition result. The character recognition result may include encrypted characters. Multiple recognitions can be performed on the same image, and the results can be merged to improve accuracy.

[0118] Among them, such as Figure 2 As shown, the system also includes a model training module 9; the model training module can be used to obtain historical single-character images and the historical character recognition results corresponding to the historical single-character images; extract the historical character features of the historical single-character images, use the historical character features as the input features of the character recognition model, use the historical character recognition results as the training labels of the character recognition model, iteratively train the character recognition model until the loss function value of the character recognition model is less than the preset loss function threshold, and determine that the character recognition model training is complete.

[0119] The decryption module 6 can decrypt the encrypted characters output by the character recognition module 5. The encrypted characters can be characters encrypted using different encryption algorithms and stored in a specific data structure, such as a byte array or a string, for subsequent processing.

[0120] One possible approach is to determine the encryption algorithm used to encrypt characters through key feature analysis. Specifically, different encryption algorithms use keys with different characteristics, such as key length and key format. The decryption module 6 can infer the encryption algorithm type based on these key features. For example, the Blowfish algorithm uses variable-length keys (typically 32-448 bits), while the AES algorithm supports 128, 192, and 256-bit key lengths. By analyzing the key length and format, the encryption algorithm type can be preliminarily determined.

[0121] One possible approach is to determine the encryption algorithm used by the encrypted characters through metadata analysis. Specifically, if the encrypted characters are stored in a specific file format or data structure, they may contain metadata information describing the encryption algorithm type, key, etc. The decryption module 6 can parse this metadata to determine the encryption type.

[0122] As one possible approach, if the encryption type is identified as a Caesar cipher, decryption module 6 can use the Caesar cipher to decrypt the encrypted characters, obtaining the decrypted characters. The Caesar cipher is a simple substitution cipher algorithm that encrypts by shifting characters a fixed number of positions in the alphabet. Decryption simply involves shifting the characters in the opposite direction by the same number of positions. For substitution ciphers, decryption module 6 can use the substitution table corresponding to the encryption method to perform the reverse substitution. Substitution ciphers encrypt by replacing each character in the alphabet with another character; decryption simply involves replacing the encrypted characters back to the original characters according to the substitution table.

[0123] If the encryption type is identified as the Blowfish algorithm, decryption module 6 can use the Blowfish decryption function in the OpenSSL library to decrypt the encrypted characters. The Blowfish algorithm uses a variable-length key, and the correct key must be provided for decryption.

[0124] If the encryption type is identified as AES, decryption module 6 can use the AES decryption function in the OpenSSL library to decrypt it. AES supports key lengths of 128, 192, and 256 bits, and requires the correct key and initialization vector for decryption.

[0125] Among them, such as Figure 2 As shown, the system also includes a notification module 10; the notification module 10 can be used to send a decryption failure notification to the user. Specifically, the success of decryption can be determined by judging whether the decrypted text conforms to the expected format and whether it contains valid characters. For example, for text data, if the decrypted text contains a large number of garbled characters or special characters and does not conform to common text encoding rules, it can be determined that the decryption has failed.

[0126] If decryption failure is detected, the alert module 10 can record relevant information about the failure, including but not limited to the encryption algorithm type, encrypted data identifier, decryption time, error code, or description. Based on user configuration or system default settings, it will send a decryption failure alert to the user. The alert method can include on-screen pop-up alerts, email alerts, and system notification alerts.

[0127] The screen pop-up notification can display a prompt window on the main control device screen, showing key information about the decryption failure, such as the encryption algorithm type, encrypted data identifier, and a brief error description, and prompting the user to view the detailed log.

[0128] Email alerts allow the system to send a notification email if the user has pre-configured an email address, alerting them of decryption failure. The email content includes detailed information about the decryption failure and a download link for the log file.

[0129] System notifications can display decryption failure information in the operating system taskbar or notification center. Users can click on the notification to quickly open the decryption failure log and view detailed information.

[0130] Output module 7 can be used to output the decrypted text output by decryption module 6. Output methods may include displaying it on the screen, saving it to a file, or transmitting it to other devices via a network.

[0131] The control module 8 can be used to sequentially call the input module 1, preprocessing module 2, character detection module 3, character segmentation module 4, character recognition module 5, decryption module 6, output module 7, model training module 9, and reminder module 10 to realize the automated process of character recognition. For example, using Python's function call mechanism, a main function can be defined in the control module to call the functions of each module in sequence according to the process.

[0132] Data formats can be standardized to ensure consistent input and output formats across modules. For example, using NumPy arrays or Pandas DataFrames can reduce the complexity of format conversion. For computationally intensive tasks (such as large-scale image batch processing), multithreading or multiprocessing techniques can be used to accelerate processing, especially in image preprocessing and character recognition stages.

[0133] During module invocation, control module 8 can monitor the running status of each module in real time, recording the module's execution time and results. If an exception occurs in a module, control module 8 can capture the exception information and handle it accordingly based on the exception type, such as retrying the operation, skipping the current processing step, and recording an error log. Simultaneously, the exception information is fed back to the user so that the user can understand the system's operating status.

[0134] like Figure 3 As shown, embodiments of this disclosure provide a character recognition method, which can be applied to the aforementioned character recognition system and executed by the control module of the character recognition system. The character recognition method may include:

[0135] Step 101: Read the original image, which is an irregularly shaped screenshot.

[0136] The original image can be an image obtained from a scanner, digital camera, smartphone, screenshot tool, network, etc.; the irregular screenshot image can be a low-quality screenshot image captured by the user through mobile devices such as mobile phones and computers, which has different sizes, tilted angles, blur, noise and / or background interference.

[0137] Step 102: Perform preprocessing operations on the original image and extract the character region image from the original image after preprocessing.

[0138] Preprocessing operations may include, but are not limited to, image correction, background removal, resizing, grayscale conversion, binarization, and image denoising. Image correction can be used to correct irregular cropped images into regular rectangular shapes; background removal can remove the image background while preserving foreground characters; resizing can adjust the image size to meet the requirements of subsequent processing modules; grayscale conversion can convert color images to grayscale images; binarization can convert grayscale images to binary images; and image denoising can remove noise points from the image. Preprocessing, through multi-dimensional image enhancement, provides high-quality input for subsequent character recognition.

[0139] A character region image can be a local image region containing character text content extracted from the original image.

[0140] In this embodiment of the disclosure, character region detection can be performed on the preprocessed original image to extract character regions from the original image and generate a character region image. To improve the accuracy of character region recognition, as a possible method, edge detection can be performed on the original image to obtain an edge image; contour detection can be performed on the edge image to obtain all contours; character contours can be selected from all contours based on character contour features; and the corresponding character region image in the original image can be extracted based on the character contours.

[0141] Step 103: Segment the character region image based on morphological operations to obtain a single character image.

[0142] Morphological operations are image processing methods based on set theory. They achieve goals such as denoising, segmentation, and filling through the interaction of structuring elements (SEs) with image pixels. Morphological operations include at least one of dilation, erosion, opening, and closing operations. A single-character image can be a binary image containing only one independent character.

[0143] Specifically, the character region image can be dilated to connect adjacent character parts; eroded to separate adhered character parts; opened to remove noise and small connected parts; closed to fill internal holes in the character region image; connected component analysis is performed on the morphologically operated character region image to obtain the connected component analysis results; and each connected component is extracted from the character region image to form a single character image based on the connected component analysis results.

[0144] Step 104: Extract character features from single-character images, input the character features into a character recognition model for character recognition processing to obtain character recognition results. The character recognition model is used to determine the probability value of the character that matches the character features. The character corresponding to the highest character probability value is determined as the character recognition result. The character recognition result includes encrypted characters.

[0145] Character recognition models can employ deep learning models, such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs).

[0146] In the embodiments of this disclosure, feature extraction algorithms, such as Histogram of Oriented Gradients (HOG), Scale Invariant Feature Transform (SIFT), and Fast Robust Feature Extraction (SURF), can be used to extract character features from a single character image. These character features can be used to describe the shape, texture, and structural information of the character.

[0147] The extracted character features can be input into a pre-trained character recognition model, which can employ deep learning models such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs). The character recognition model can learn from a large number of character samples to identify the character category corresponding to the input features.

[0148] The character recognition model outputs the probability value of each character category matching the input features. This probability value is calculated using the softmax function and represents the likelihood of the input features belonging to each character category. The character with the highest probability value is then identified as the character recognition result.

[0149] Step 105: Decrypt the encrypted characters and output the decrypted text.

[0150] In this embodiment of the disclosure, since the character recognition result includes encrypted characters, further decryption processing is required.

[0151] Based on the encryption algorithm and key, encrypted characters can be decrypted to obtain the decrypted characters. The decryption method can be the inverse operation of the encryption algorithm, such as the decryption process of a symmetric encryption algorithm. The decrypted characters are then output as the final decrypted text. This decrypted text can be used in various applications, such as information display, data storage, and text editing.

[0152] In summary, according to the character recognition method provided in this disclosure, compared with the existing technology, this disclosure can read the original image through an input module, wherein the original image is an irregular screenshot image; a preprocessing module performs preprocessing operations on the original image; a character detection module extracts character region images from the preprocessed original image; a character segmentation module segments the character region images based on morphological operations to obtain single character images; a character recognition module extracts the character features of the single character images, inputs the character features into a character recognition model for character recognition processing to obtain character recognition results, wherein the character recognition model is used to determine the character probability value that matches the character features, and the character corresponding to the highest character probability value is determined as the character recognition result, wherein the character recognition result includes encrypted characters; a decryption module decrypts the encrypted characters; an output module outputs decrypted text; and a control module sequentially calls each module to realize the automated process of character recognition.

[0153] The solution disclosed herein allows for an automated process, from raw image input to decrypted text output, orchestrated by a control module without human intervention. When processing large numbers of irregularly shaped screenshots, the system can automatically complete image reading, preprocessing, character detection, segmentation, recognition, decryption, and output, reducing manual operation time and error rates, and significantly improving processing efficiency.

[0154] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0155] Obviously, those skilled in the art should understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by the computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A character recognition system, characterized in that, include: The system includes an input module, a preprocessing module, a character detection module, a character segmentation module, a character recognition module, a decryption module, an output module, and a control module. The input module, the preprocessing module, the character detection module, the character segmentation module, the character recognition module, the decryption module, and the output module are all connected to the control module. The input module is used to read the original image, which is an irregularly shaped screenshot. The preprocessing module is used to perform preprocessing operations on the original image; The character detection module is used to extract character region images from the original image after preprocessing. The character segmentation module is used to segment the character region image based on morphological operations to obtain a single character image; The character recognition module is used to extract character features from the single character image, input the character features into the character recognition model for character recognition processing to obtain character recognition results, the character recognition model is used to determine the character probability value that matches the character features, and determine the character corresponding to the highest character probability value as the character recognition result, the character recognition result includes encrypted characters; The decryption module is used to decrypt the encrypted characters; The output module is used to output the decrypted text; The control module is used to call each module in sequence to realize the automated process of character recognition.

2. The character recognition system according to claim 1, characterized in that, The preprocessing includes image correction processing; The preprocessing module is specifically used to: determine the corner coordinates of the original image and the vertex coordinates of the target image; determine the geometric transformation matrix based on the corner coordinates and the vertex coordinates; and use the geometric transformation matrix to geometrically transform the original image into the target image, wherein the target image is a regular screenshot image obtained by geometric transformation of the original image, and the geometric transformation matrix is ​​used to describe the coordinate mapping relationship from the original image to the target image.

3. The character recognition system according to claim 1, characterized in that, The preprocessing includes image denoising; The preprocessing module is specifically used for: identifying Gaussian noise and / or salt-and-pepper noise in the original image; performing denoising processing on the original image using nonlocal mean filtering to obtain a first filtered image; performing smoothing processing on the original image using bilateral filtering to obtain a second filtered image; performing weighted fusion operation on the first filtered image and the second filtered image to suppress the Gaussian noise; and / or, performing noise point probability prediction on each pixel in the original image to obtain the probability value of each pixel being a salt-and-pepper noise point, and performing median filtering processing on pixels with probability values ​​greater than a preset probability threshold to suppress the salt-and-pepper noise.

4. The character recognition system according to claim 1, characterized in that, The preprocessing includes binarization; The preprocessing module is specifically used to: traverse each pixel in the original image and compare the pixel value of each pixel with a preset pixel threshold; set pixels with pixel values ​​greater than the preset pixel threshold to white; and set pixels with pixel values ​​less than or equal to the preset pixel threshold to black.

5. The character recognition system according to claim 1, characterized in that, The character detection module is specifically used for: performing edge detection on the original image to obtain an edge image; performing contour detection on the edge image to obtain all contours; filtering character contours from all contours based on character contour features; and extracting the corresponding character region image from the original image based on the character contours.

6. The character recognition system according to claim 1, characterized in that, The morphological operations include at least one of dilation, erosion, opening, and closing operations; The character segmentation module is specifically used to: perform a dilation operation on the character region image to connect adjacent character parts; perform an erosion operation on the character region image to separate adhered character parts; and perform an opening operation on the character region image to remove noise and small connected parts. A closing operation is performed on the character region image to fill the holes inside the characters; The character region image after morphological operations is subjected to connected component analysis to obtain the connected component analysis results. Based on the connected component analysis results, each connected component is extracted from the character region image to form the single character image.

7. The character recognition system according to claim 1, characterized in that, The system also includes a model training module; The model training module is used to acquire historical single-character images and the corresponding historical character recognition results; extract historical character features from the historical single-character images, use the historical character features as input features of the character recognition model, use the historical character recognition results as training labels of the character recognition model, iteratively train the character recognition model until the loss function value of the character recognition model is less than a preset loss function threshold, and determine that the character recognition model training is complete.

8. The character recognition system according to claim 1, characterized in that, The decryption module is specifically used to: decrypt the encrypted characters using a Caesar cipher or a substitution cipher; The system also includes: a reminder module; The reminder module is used to record relevant information about the decryption failure and send a decryption failure reminder to the user if the decryption fails.

9. The character recognition system according to claim 1, characterized in that, Each module is encapsulated as an independent function or class.

10. A character recognition method, characterized in that, The method is applied to the character recognition system as described in any one of claims 1-9, and the method includes: Read the original image, which is an irregularly shaped screenshot; The original image is preprocessed, and the character region image is extracted from the original image after the preprocessing operation. The character region image is segmented based on morphological operations to obtain a single character image; The character features of the single-character image are extracted, and the character features are input into a character recognition model for character recognition processing to obtain a character recognition result. The character recognition model is used to determine the probability value of the character that matches the character features. The character corresponding to the highest probability value is determined as the character recognition result. The character recognition result includes encrypted characters. The encrypted characters are decrypted, and the decrypted text is output.

Images