Document image correction method and apparatus, electronic device, and readable medium

Abstract

The application discloses a document image correction method and device, electronic equipment and readable medium, the document image correction method comprises: text detection and text recognition are carried out on the to-be-corrected document image, first text detection result and first text recognition result are obtained, and the first confidence of first text recognition result is calculated; when the first confidence is less than or equal to the first threshold value, the arrangement direction of the text character bar of the to-be-corrected document image is determined based on the first text detection result and the first text recognition result, and the to-be-corrected document image is rotated based on the arrangement direction; text recognition is carried out on the to-be-corrected document image after rotation, second text recognition result is obtained, second confidence of second text recognition result is calculated, and the to-be-corrected document image is corrected based on the first confidence and the second confidence. The document image correction method provided by the application can correct the to-be-corrected document image to its standard direction, so as to improve the OCR recognition accuracy of the document image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a document image correction method and device, an electronic device and a readable medium. BACKGROUND

[0002] In the prior art, the text content in a document image is generally recognized by an OCR (Optical Character Recognition) technology. When performing OCR recognition, the placement direction of the document image is usually required to match the recognition direction of the OCR, otherwise the OCR recognition result is likely to be incorrect, resulting in a large number of garbled codes.

[0003] Therefore, in view of the above technical problems, it is necessary to provide a document image correction method and device, an electronic device and a readable medium. SUMMARY

[0004] The present application aims to provide a document image correction method and device, an electronic device and a readable medium, which can correct the direction of a document image to improve the OCR recognition accuracy of the document image.

[0005] To achieve the above-mentioned purpose, the technical solution provided by the present application is as follows:

[0006] In a first aspect, the present application provides a document image correction method, comprising:

[0007] performing text detection and text recognition on the document image to be corrected to obtain a first text detection result and a first text recognition result, and calculating a first confidence of the first text recognition result; when the first confidence is less than or equal to a first threshold, determining the arrangement direction of the text lines in the document image to be corrected based on the first text detection result and the first text recognition result, and rotating the document image to be corrected based on the arrangement direction; performing text recognition on the rotated document image to be corrected to obtain a second text recognition result, calculating a second confidence of the second text recognition result, and correcting the document image to be corrected based on the first confidence and the second confidence.

[0008] In one or more embodiments, performing text detection and text recognition on the document image to be corrected to obtain the first text detection result and the first text recognition result, and calculating the first confidence of the first text recognition result, comprises: performing text detection on the document image to be corrected to obtain text lines in the document image to be corrected; performing text recognition on the text lines to obtain the first text recognition result; obtaining probability values of each character in the first text recognition result, and calculating the first confidence of the first text recognition result based on the probability values.

[0009] In one or more embodiments, the first confidence of the first text recognition result is calculated based on the probability values, including: calculating an average of the probability values of all characters in the first text recognition result, and taking the average as the first confidence; or selecting N characters with larger probability values from all characters in the first text recognition result, calculating an average of the probability values of the selected N characters, and taking the average as the first confidence; where N is a positive integer and the ratio of N to the total number of all characters in the first text recognition result is not less than one half.

[0010] In one or more embodiments, the arrangement direction of the text lines in the document image to be corrected is determined based on the first text detection result and the first recognition result, including: determining the proportion of single-character text boxes relative to the total text box in the document image to be corrected based on the first text detection result and the first recognition result; when the proportion is greater than a second threshold, the arrangement direction of the text lines in the document image to be corrected is determined as vertical, and the document image to be corrected is rotated by 90°; when the proportion is less than or equal to the second threshold, the arrangement direction of the text lines in the document image to be corrected is determined as horizontal, and the document image to be corrected is rotated by 180°.

[0011] In one or more embodiments, text recognition is performed on the rotated document image to be corrected to obtain a second text recognition result, a second confidence of the second text recognition result is calculated, and the document image to be corrected is corrected based on the first confidence and the second confidence, including: performing text recognition on the document image to be corrected rotated by 180° to obtain a second text recognition result, calculating a second confidence of the second text recognition result, and determining whether the second confidence is greater than the first confidence; when the second confidence is greater than the first confidence, the correction of the document image to be corrected is ended; when the second confidence is less than or equal to the first confidence, the document image to be corrected rotated by 180° is rotated by 180° to correct the document image to be corrected.

[0012] In one or more embodiments, text recognition is performed on the rotated document image to be corrected to obtain a second text recognition result, a second confidence of the second text recognition result is calculated, and the document image to be corrected is corrected based on the first confidence and the second confidence, including: performing text recognition on the document image to be corrected rotated by 90° to obtain a second text recognition result, calculating a second confidence of the second text recognition result, and determining whether the second confidence is greater than the first threshold; when the second confidence is greater than the first threshold, the correction of the document image to be corrected is ended.

[0013] In one or more embodiments, the method further comprises: when the second confidence is less than or equal to the first threshold, rotating the rotated 90° to-be-corrected document image by 180°; performing text recognition on the rotated 180° to-be-corrected document image to obtain a third text recognition result, calculating a third confidence of the third text recognition result, and determining whether the third confidence is greater than the second confidence; when the third confidence is greater than the second confidence, ending the correction of the to-be-corrected document image; and when the third confidence is less than or equal to the second confidence, rotating the rotated 180° to-be-corrected document image by 180° to correct the to-be-corrected document image.

[0014] In one or more embodiments, the method further comprises: when the first confidence is greater than a first threshold, ending the correction of the to-be-corrected document image.

[0015] In a second aspect, the present application provides a document image correction device, comprising:

[0016] a calculation module configured to perform text detection and text recognition on a to-be-corrected document image to obtain a first text detection result and a first text recognition result, and calculate a first confidence of the first text recognition result; a rotation module configured to, when the first confidence is less than or equal to a first threshold, determine an arrangement direction of text lines in the to-be-corrected document image based on the first text detection result and the first text recognition result, and rotate the to-be-corrected document image based on the arrangement direction; and a correction module configured to perform text recognition on the rotated to-be-corrected document image to obtain a second text recognition result, calculate a second confidence of the second text recognition result, and correct the to-be-corrected document image based on the first confidence and the second confidence.

[0017] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the document image correction method as described above.

[0018] In a fourth aspect, the present application provides a computer readable medium, wherein the computer readable medium carries computer execution instructions, and the computer execution instructions are executed by a processor to implement the document image correction method as described above.

[0019] Compared with the prior art, the document image correction method provided by the application can correct the to-be-corrected document image to its standard direction, so as to improve the OCR recognition accuracy of the document image, by performing text detection and text recognition on the to-be-corrected document image to obtain a first text detection result and a first text recognition result, calculating a first confidence of the first text recognition result, and correcting the to-be-corrected document image based on the relationship between the first confidence and a first threshold. The method can directly use the text detection model and the text recognition model results in the OCR system without training other models, can save computer resources, and can achieve high correction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is an application scenario diagram of the document image correction method of an embodiment of the application;

[0021] Figure 2 is a flowchart of the document image correction method of an embodiment of the application;

[0022] Figure 3 is a structural block diagram of the document image correction device of an embodiment of the application;

[0023] Figure 4 is a structural block diagram of an electronic device of an embodiment of the application. DETAILED DESCRIPTION

[0024] The specific embodiments of the application will be described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the application is not limited by the specific embodiments.

[0025] Unless otherwise clearly indicated, in the entire specification and claims, the term "comprise" or its variants such as "contain" or "include" and the like will be understood to include the stated element or component, but not to exclude other elements or components.

[0026] In order to facilitate the understanding of the technical solutions of the present application, the technical terms that may appear in the present application will be explained in detail below.

[0027] Document image: refers to an image containing text content in a document. For example, it can be an image in a PDF document formed by photographing or scanning a paper text. For the document image in the PDF document, each page can be regarded as a document image.

[0028] Standard direction: refers to a direction matching the recognition direction of the recognition system. Generally, the document image is input into the recognition system in the standard direction, so that a higher recognition accuracy can be obtained. Generally, the document image in the standard direction is more in line with human reading habits.

[0029] OCR (Optical Character Recognition): refers to the process of using electronic devices to check characters printed on paper, determining their shape by detecting light and dark patterns, and then translating the shape into computer text using character recognition methods; that is, for printed characters, using optical methods to convert the characters in paper documents into black and white dot matrix document images, and through computer processing and analysis of the images, the characters or text in the images are converted into editable text form technology.

[0030] The OCR system needs to convert the characters or text in the text area of the document image into editable text form, and the recognition direction of the document image directly affects the recognition accuracy and efficiency of the OCR system. If the recognition direction of the document image is incorrect, the OCR system may recognize the characters or text into the wrong form, or may not be able to recognize it. For example, in a Chinese OCR system, if the recognition direction of the image is incorrect, the characters may be recognized as incorrect characters.

[0031] In order to avoid the foregoing problems, the document image correction method and device provided by the present application can be used for the correction of the document image, so that it has a correct recognition direction.

[0032] Please refer to Figure 1 Fig. 1 is a schematic diagram of an exemplary application scenario of the document image correction method provided by the present application. In Figure 1 The implementation scenario shown in Fig. 1 includes a client 101, a document image correction server 102 and a network 103. The network 103 is a medium for providing a communication link between the client 101 and the document image correction server 102. The network 103 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc., and the network 103 can be at least one of a local area network, a metropolitan area network and a wide area network.

[0033] The client 101 can be an electronic device for providing a document image to be corrected. For example, the electronic device can be a mobile terminal such as a smartphone, a tablet computer, a laptop computer, etc., or a terminal such as a desktop computer, a projection computer, etc., and the present application does not limit the same.

[0034] The document image correction server 102 refers to a server for running any document image correction program and providing corresponding document image correction service. The document image correction server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and big data and artificial intelligence platform.

[0035] The user can upload the to-be-corrected document image to the document image correction server 102 through the client 101, and the document image correction server 102 performs text detection and text recognition on the to-be-corrected document image to obtain the first text detection result and the first text recognition result, and calculates the first confidence of the first text recognition result; and then corrects the to-be-corrected document image based on the first confidence. When the first confidence is less than or equal to a first threshold, the document image correction server 102 can determine the arrangement direction of the text lines of the to-be-corrected document image based on the first text detection result and the first text recognition result, and rotate the to-be-corrected document image based on the arrangement direction. The document image correction server 102 can perform text recognition on the rotated to-be-corrected document image to obtain a second text recognition result, calculate a second confidence of the second text recognition result, and correct the to-be-corrected document image based on the first confidence and the second confidence.

[0036] Please refer to Figure 2 The document image correction method specifically includes the following steps:

[0037] S201: Perform text detection and text recognition on the to-be-corrected document image to obtain the first text detection result and the first text recognition result, and calculate the first confidence of the first text recognition result.

[0038] It should be noted that the text detection on the to-be-corrected document image refers to detecting the text region in the document image, separating the text region and the background region, and preparing for subsequent text recognition. Text detection usually involves the following processes: image preprocessing, denoising, grayscale, binarization and other processing on the input image to better detect the text region; text line detection, using image processing and computer vision techniques to detect and segment the text in the image to form a text box, commonly used text line detection algorithms include edge detection, region segmentation, template matching based methods, etc. A deep neural network model can be used to implement the text detection function, such as CTPN (Connectionist Text Proposal Network), PSENet (Shape Robust Text Detection with Progressive Scale Expansion Network), DBNet (Differentiable Binarization Network) model, etc.

[0039] Text recognition refers to character segmentation and character recognition of the detected text line, and converting characters in the text image into editable text. Text recognition usually includes the following processes: character segmentation, which separates characters in the text line, character segmentation is a key step in text recognition, which needs to separate different characters or text for subsequent character recognition; character recognition, which recognizes the segmented characters or text, character recognition needs to use computer vision, machine learning, deep learning and other technologies to extract features and classify characters to obtain higher recognition accuracy and efficiency; post-processing, which processes the recognition result to obtain more accurate text recognition result, post-processing methods include error correction, spelling check, recognition result correction, etc. A neural network model can be used to implement the text recognition function, such as CRNN (Convolutional Recurrent Neural Network), RARE (Robust text recognizer with Automatic REctification) model, etc.

[0040] In an example embodiment, the text detection and the text recognition are performed on the document image to be corrected to obtain a first text detection result and a first text recognition result, and the first confidence of the first text recognition result is calculated in the following manner: text detection is performed on the document image to be corrected to obtain text lines in the document image to be corrected; text recognition is performed on the text lines to obtain the first text recognition result; probability values of each character in the first text recognition result are obtained, and the first confidence of the first text recognition result is calculated based on the probability values.

[0041] It should be noted that when the neural network model is used for text recognition, CNN (Convolutional Neural Network) can be used to extract features from the image, so as to divide the extracted image features into a plurality of feature sequences. These feature sequences can be used as the input of the RNN (Recurrent Neural Network) recurrent neural network model, and the probability of belonging to each character category at each time step is predicted through the recurrent neural network. These categories include Chinese characters, numbers, letters, symbols, etc. The RNN model generally uses the CTC (Connectionist Temporal Classification) algorithm to decode the final recognized text.

[0042] In an example embodiment, the first confidence of the first text recognition result is calculated based on the probability values in the following manner: the average value of the probability values of all characters in the first text recognition result is calculated, and the average value is used as the first confidence.

[0043] It can be understood that after the text recognition is performed by the foregoing text recognition model, the probability values of all characters in the first text recognition result can be obtained, and the average value of the probability values of all characters can be calculated by adding the probability values of all characters and dividing the sum by the total number of all characters. The average value is the first confidence of the first text recognition result.

[0044] For example, in an embodiment, the total number of characters in the first text recognition result is 10, and the probability values of the characters are 80%, 76%, 88%, 63%, 75%, 92%, 90%, 84%, 90%, and 86% respectively. The average value of the probability values of all characters can be calculated by adding the probability values of the characters and dividing the sum by the total number of characters 10, which is 82.4%. That is, the first confidence of the first text recognition result is 82.4%.

[0045] In an example embodiment, the manner of calculating the first confidence of the first text recognition result based on the probability values specifically comprises: selecting N characters with larger probability values from all characters in the first text recognition result, calculating an average value of the probability values of the selected N characters, and taking the average value as the first confidence. Here, N is a positive integer, and the ratio of N to the total number of all characters in the first text recognition result is preferably not less than one half.

[0046] It should be noted that, in the process of text recognition on the text lines, there can be unclear characters or rare characters in the recognized text lines. In general, the character probability values corresponding to the recognition results of these characters are relatively low. In order to avoid affecting the reliability of the first confidence in the case that there are many unclear characters or rare characters in the text lines, N characters with larger probability values can be selected from all characters in the first text recognition result to calculate the probability average value, and the average value is taken as the first confidence.

[0047] When N characters with larger probability values are selected from all characters in the first text recognition result to calculate the probability average value, in order to further improve the reliability of the first confidence, the ratio of N to the total number of all characters in the first text recognition result is preferably not less than one half. When the ratio of N to the total number of all characters in the first text recognition result is less than one half, the first confidence can be unreliable due to too few selected characters.

[0048] S202: When the first confidence is less than or equal to a first threshold value, the arrangement direction of the text lines in the to-be-corrected document image is determined based on the first text detection result and the first text recognition result, and the to-be-corrected document image is rotated based on the arrangement direction.

[0049] It should be noted that the first threshold value is a threshold value for measuring the reliability of the first text recognition result. In general, if the first confidence of the first text recognition result is greater than the first threshold value, it can be considered that the first text recognition result is reliable. Otherwise, it can be considered that the first text recognition result is unreliable. The first threshold value can be set according to actual needs. In general, different text recognition manners and different calculation manners of the first confidence can result in different first threshold values.

[0050] In an example embodiment, the manner of determining the arrangement direction of the text lines in the to-be-corrected document image based on the first text detection result and the first text recognition result specifically comprises: determining a proportion of single-character text boxes relative to total text boxes in the to-be-corrected document image based on the first text detection result and the first text recognition result; when the proportion is greater than a second threshold, the arrangement direction of the text lines in the to-be-corrected document image is determined as vertical; and when the proportion is less than or equal to the second threshold, the arrangement direction of the text lines in the to-be-corrected document image is determined as horizontal.

[0051] It can be understood that a document image (page) in a document (such as a PDF document) usually includes four placement directions, i.e., 0° (same as a standard direction), 90° (offset by 90° relative to the standard direction), 180° (offset by 180° relative to the standard direction), and 270° (offset by 270° relative to the standard direction). Among them, the 0° direction is the standard direction matching the OCR recognition direction, and the OCR recognition result obtained in this direction is more accurate. The offset angle can be an angle of clockwise rotation relative to the standard direction, or an angle of counterclockwise rotation.

[0052] It should be noted that in the present embodiment, the horizontal direction refers to a direction parallel to the arrangement direction of the text lines in the document image when the offset angle of the document image is 0° or 180°. The vertical direction refers to a direction parallel to the arrangement direction of the text lines in the document image when the offset angle of the document image is 90° or 270°. Text detection can frame the text area of the document image, and text recognition can recognize the framed text area (text box) to determine how many characters are in the framed text area.

[0053] It should also be noted that the text detection model used for text detection is trained using horizontal (i.e., text lines are arranged horizontally) text samples, and the second threshold is a standard threshold for judging the arrangement direction of the text lines. When the offset angle of the document image is 90° or 270°, most of the text detection results are single-character text boxes. When the offset angle of the document image is 0° or 180°, fewer single-character text boxes appear in the text detection result. Therefore, the arrangement direction of the text lines in the document image can be determined according to the proportion of the number of single-character text boxes relative to the total text boxes in the text detection result.

[0054] Specifically, the manner of rotating the to-be-corrected document image based on the arrangement direction specifically comprises: when the arrangement direction of the text lines in the to-be-corrected document image is determined as horizontal, rotating the to-be-corrected document image by 180°; and when the arrangement direction of the text lines in the to-be-corrected document image is determined as vertical, rotating the to-be-corrected document image by 90°.

[0055] It can be understood that for the to-be-corrected document image with the arrangement direction of text lines being horizontal, after being rotated by 180°, the to-be-corrected document image can be offset by 0° or 180° relative to the standard direction. For the to-be-corrected document image with the arrangement direction of text lines being vertical, after being rotated by 90°, the to-be-corrected document image can be offset by 0° or 180° relative to the standard direction.

[0056] S203: performing text recognition on the rotated to-be-corrected document image, obtaining a second text recognition result, calculating a second confidence degree of the second text recognition result, correcting the to-be-corrected document image based on the first confidence degree and the second confidence degree.

[0057] It should be noted that the manner of performing text recognition on the rotated to-be-corrected document image and the manner of calculating the second confidence degree in step S203 can be the same as or different from step S201. When the manner of performing text recognition in step 203 or the manner of calculating the second confidence degree is the same as step S201, the calculated second confidence degree can be directly compared with the first confidence degree. When the manner of performing text recognition in step 203 or the manner of calculating the second confidence degree is different from step S201, the calculated second confidence degree or the first confidence degree can be appropriately modified so that they can be compared under the same standard.

[0058] In an exemplary embodiment, text recognition is performed on the to-be-corrected document image rotated by 180° in step S202, a second text recognition result is obtained, a second confidence degree of the second text recognition result is calculated, and it is determined whether the second confidence degree is greater than the first confidence degree. When the second confidence degree is greater than the first confidence degree, the correction of the to-be-corrected document image is ended. When the second confidence degree is less than or equal to the first confidence degree, the to-be-corrected document image rotated by 180° is rotated by 180° to correct the to-be-corrected document image.

[0059] It can be understood that, for the to-be-corrected document image rotated 180° in step S202, when the second confidence is greater than the first confidence, it indicates that the confidence of the second text recognition result is higher than the confidence of the first text recognition result, that is, the document image recognition direction corresponding to the second text recognition result is a more accurate recognition direction, at this time, the current placement direction of the to-be-corrected document image rotated 180° can be considered as the standard direction thereof, and no further correction is needed. When the second confidence is less than or equal to the first confidence, it indicates that the confidence of the second text recognition result is lower than the confidence of the first text recognition result, that is, the document image recognition direction corresponding to the first text recognition result is a more accurate recognition direction, at this time, the to-be-corrected document image rotated 180° needs to be rotated 180° again to restore to the recognition direction corresponding to the first text recognition result, so as to correct the to-be-corrected document image.

[0060] In an exemplary embodiment, text recognition is performed on the to-be-corrected document image rotated 90° in step S202 to obtain a second text recognition result, a second confidence of the second text recognition result is calculated, and it is judged whether the second confidence is greater than the first threshold. When the second confidence is greater than the first threshold, the correction of the to-be-corrected document image is ended.

[0061] It can be understood that, for the to-be-corrected document image rotated 90° in step S202, when the second confidence is greater than the first threshold, it indicates that the second text recognition result is reliable, that is, the document image recognition direction corresponding to the second text recognition result is an accurate recognition direction, at this time, the current placement direction of the to-be-corrected document image rotated 90° can be considered as the standard direction thereof, and no further correction is needed.

[0062] In an exemplary embodiment, for the to-be-corrected document image rotated 90° in step S202, when the second confidence is less than or equal to the first threshold, the to-be-corrected document image rotated 90° is rotated 180°. Text recognition is performed on the to-be-corrected document image rotated 180° to obtain a third text recognition result, a third confidence of the third text recognition result is calculated, and it is judged whether the third confidence is greater than the second confidence.

[0063] It can be understood that, for the to-be-corrected document image rotated 90° in step S202, when the second confidence is less than or equal to the first threshold, it indicates that the second text recognition result is unreliable, at this time, the to-be-corrected document image rotated 90° needs to be rotated 180° again, and further text recognition is performed to obtain a third text recognition result, so as to calculate a third confidence.

[0064] Specifically, when the third confidence is greater than the second confidence, the correction of the to-be-corrected document image is ended. When the third confidence is less than or equal to the second confidence, the to-be-corrected document image rotated by 180° is rotated by 180° to correct the to-be-corrected document image.

[0065] It can be understood that, for the to-be-corrected document image rotated by 90° and then rotated by 180° in step S203, when the third confidence is greater than the second confidence, it indicates that the confidence of the third text recognition result is higher than the confidence of the second text recognition result, that is, the document image recognition direction corresponding to the third text recognition result is a more accurate recognition direction, at this time, the current placement direction of the to-be-corrected document image rotated by 180° again can be considered as the standard direction, and no further correction is needed. When the third confidence is less than or equal to the second confidence, it indicates that the confidence of the third text recognition result is lower than the confidence of the second text recognition result, that is, the document image recognition direction corresponding to the second text recognition result is a more accurate recognition direction, at this time, the to-be-corrected document image rotated by 180° needs to be rotated by 180° again to restore it to the recognition direction corresponding to the second text recognition result, so as to correct the to-be-corrected document image.

[0066] S204: When the first confidence is greater than the first threshold value, the correction of the to-be-corrected document image is ended.

[0067] It can be understood that, when the first confidence of the first text recognition result is greater than the first threshold value, it indicates that the first text recognition result is reliable, that is, the document image recognition direction corresponding to the first text recognition result is an accurate recognition direction, at this time, the current placement direction of the to-be-corrected document image at this time can be considered as the standard direction, and no further correction is needed.

[0068] In summary, the document image correction method provided by the present application can correct the to-be-corrected document image to its standard direction by performing text detection and text recognition on the to-be-corrected document image, obtaining the first text detection result and the first text recognition result, calculating the first confidence of the first text recognition result, and correcting the to-be-corrected document image based on the relationship between the first confidence and the first threshold value, so as to improve the OCR recognition accuracy of the document image. The method can directly use the text detection model and the text recognition model results in the OCR system without the need for additional training of other models, can save computer resources, and can achieve a high correction accuracy.

[0069] Please refer to Figure 3 Based on the same inventive concept as the foregoing document image correction method, an embodiment of the present application provides a document image correction device 300, which comprises a calculation module 301, a rotation module 302, and a correction module 303.

[0070] The computing module 301 is configured to perform text detection and text recognition on the to-be-corrected document image, obtain the first text detection result and the first text recognition result, and calculate a first confidence of the first text recognition result. The rotating module 302 is configured to, when the first confidence is less than or equal to a first threshold, determine an arrangement direction of the text lines in the to-be-corrected document image based on the first text detection result and the first text recognition result, and rotate the to-be-corrected document image based on the arrangement direction. The correcting module 303 is configured to perform text recognition on the rotated to-be-corrected document image, obtain a second text recognition result, calculate a second confidence of the second text recognition result, and correct the to-be-corrected document image based on the first confidence and the second confidence.

[0071] Specifically, the computing module 301 can be configured to perform text detection on the to-be-corrected document image to obtain text lines in the to-be-corrected document image, perform text recognition on the text lines to obtain the first text recognition result, obtain probability values of each character in the first text recognition result, and calculate a first confidence of the text recognition result based on the probability values.

[0072] Specifically, the rotating module 302 can be configured to determine a proportion of single-character text boxes relative to a total text box in the to-be-corrected document image based on the first text detection result and the first text recognition result, determine the arrangement direction of the text lines in the to-be-corrected document image as vertical when the proportion is greater than a second threshold, and determine the arrangement direction of the text lines in the to-be-corrected document image as horizontal when the proportion is less than or equal to the second threshold.

[0073] Further, the rotating module 302 can be further configured to rotate the to-be-corrected document image by 180° when the arrangement direction of the text lines in the to-be-corrected document image is determined as horizontal, and rotate the to-be-corrected document image by 90° when the arrangement direction of the text lines in the to-be-corrected document image is determined as vertical.

[0074] Specifically, the correcting module 303 can be configured to perform text recognition on the to-be-corrected document image rotated by 180° to obtain a second text recognition result, calculate a second confidence of the second text recognition result, determine whether the second confidence is greater than the first confidence, end the correction of the to-be-corrected document image when the second confidence is greater than the first confidence, and rotate the to-be-corrected document image rotated by 180° by 180° to correct the to-be-corrected document image when the second confidence is less than or equal to the first confidence.

[0075] Further, the correction module 303 can also be configured to perform text recognition on the to-be-corrected document image rotated by 90°, obtain a second text recognition result, calculate a second confidence of the second text recognition result, and determine whether the second confidence is greater than the first threshold. When the second confidence is greater than the first threshold, the correction of the to-be-corrected document image is ended.

[0076] Further, the correction module 303 can also be configured to, when the second confidence is less than or equal to the first threshold, rotate the to-be-corrected document image rotated by 90° by 180°, perform text recognition on the to-be-corrected document image rotated by 180°, obtain a third text recognition result, calculate a third confidence of the third text recognition result, and determine whether the third confidence is greater than the second confidence. When the third confidence is greater than the second confidence, the correction of the to-be-corrected document image is ended. When the third confidence is less than or equal to the second confidence, the to-be-corrected document image rotated by 180° is rotated by 180° to correct the to-be-corrected document image.

[0077] Please refer to Figure 4 The electronic device 400 provided by the embodiment of the present application includes at least one processor 401, a memory 402 (for example, a non-volatile memory), an internal memory 403, and a communication interface 404, and the at least one processor 401, the memory 402, the internal memory 403, and the communication interface 404 are connected together via a bus 405. The at least one processor 401 is configured to invoke at least one program instruction stored or encoded in the memory 402, so as to enable the at least one processor 401 to perform various operations and functions of the document image correction method described in various embodiments of the present application.

[0078] In the embodiments of the present application, the electronic device 400 can include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile electronic device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable electronic device, a consumer electronic device, and the like.

[0079] The embodiment of the present application also provides a computer readable medium, and the computer readable medium carries computer execution instructions. When the computer execution instructions are executed by a processor, various operations and functions of the document image correction method described in various embodiments of the present application can be implemented.

[0080] The computer readable medium in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0081] In the present application, the computer readable signal medium can include a data signal propagated in a baseband or propagated as a carrier wave in a propagated data signal, in which a computer readable program code is embodied. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0082] Those skilled in the art should clearly understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, systems, and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the flow of the flowcharts and / or block diagrams. The computer program instructions can also be stored in a computer readable medium that can guide the computer or other programmable data processing apparatus to work in a specific manner, so that the computer program instructions stored in the computer readable medium produce a device that implements the flow of the flowcharts and / or block diagrams.Figure 1 apparatus for performing the function specified by the one or more blocks Figure 1 apparatus for performing the function specified by the one or more blocks

[0084] The foregoing description of specific exemplary embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed, and various modifications and variations are possible in light of the above teachings. It is intended that the application embrace all alternatives, modifications, and variations as can come within the scope of the description and claims. It is intended that the scope of the application be limited not with this detailed description, but rather determined by the claims and their equivalents.

Claims

1. A document image correction method, characterized in that, include: Perform text detection and text recognition on the document image to be corrected, and obtain the first text detection result and the first text recognition result; Obtain the probability value of each character in the first text recognition result, and calculate the first confidence level of the first text recognition result based on the probability value; When the first confidence level is less than or equal to the first threshold, based on the first text detection result and the first text recognition result, the arrangement direction of the text bars in the document image to be corrected is determined, and the document image to be corrected is rotated based on the arrangement direction; wherein, determining the arrangement direction based on the first text detection result and the first text recognition result includes: determining the proportion of single-character text boxes relative to the total text boxes in the first text detection result; when the proportion is greater than the second threshold, the arrangement direction is determined to be vertical and the document image to be corrected is rotated by 90°; when the proportion is less than or equal to the second threshold, the arrangement direction is determined to be horizontal and the document image to be corrected is rotated by 180°; Text recognition is performed on the rotated document image to be corrected to obtain a second text recognition result. A second confidence level is calculated based on the probability value of each character in the second text recognition result. Based on the first confidence level and the second confidence level, the document image to be corrected is corrected. The method of correcting the document image to be corrected based on the first confidence level and the second confidence level includes: performing text recognition on the document image to be corrected after rotating it by 180°, obtaining a second text recognition result, calculating a second confidence level of the second text recognition result, and determining whether the second confidence level is greater than the first confidence level; when the second confidence level is greater than the first confidence level, ending the correction of the document image to be corrected; when the second confidence level is less than or equal to the first confidence level, rotating the document image to be corrected by 180° by 180° to correct the document image to be corrected.

2. The document image correction method as described in claim 1, characterized in that, Perform text detection and text recognition on the document image to be corrected, and obtain the first text detection result and the first text recognition result, including: Perform text detection on the document image to be corrected to obtain the text entries in the document image to be corrected; The text message is subjected to text recognition to obtain the first text recognition result.

3. The document image correction method as described in claim 1, characterized in that, Calculating the first confidence level of the first text recognition result based on the probability value includes: Calculate the average probability value of all characters in the first text recognition result, and use the average value as the first confidence level; or Select N characters with higher probability values ​​from all characters in the first text recognition result, calculate the average probability value of the selected N characters, and use the average value as the first confidence level; wherein, N is a positive integer and the ratio of N to the total number of characters in the first text recognition result is not less than one-half.

4. The document image correction method as described in claim 1, characterized in that, Perform text recognition on the rotated document image to be corrected to obtain a second text recognition result, calculate a second confidence level of the second text recognition result, and correct the document image to be corrected based on the first confidence level and the second confidence level, including: Text recognition is performed on the document image to be corrected after being rotated 90° to obtain a second text recognition result. The second confidence level of the second text recognition result is calculated, and it is determined whether the second confidence level is greater than the first threshold. When the second confidence level is greater than the first threshold, the correction of the document image to be corrected ends.

5. The document image correction method as described in claim 4, characterized in that, The method further includes: When the second confidence level is less than or equal to the first threshold, the document image to be corrected after being rotated 90° is rotated 180°. Text recognition is performed on the document image to be corrected after being rotated 180° to obtain a third text recognition result. The third confidence level of the third text recognition result is calculated, and it is determined whether the third confidence level is greater than the second confidence level. When the third confidence level is greater than the second confidence level, the correction of the document image to be corrected ends; When the third confidence level is less than or equal to the second confidence level, the document image to be corrected after being rotated 180° is rotated 180° to correct the document image to be corrected.

6. The document image correction method as described in claim 1, characterized in that, The method further includes: When the first confidence level is greater than the first threshold, the correction of the document image to be corrected is terminated.

7. A document image correction apparatus for implementing the document image correction method according to any one of claims 1 to 6, characterized in that, include: The calculation module is used to perform text detection and text recognition on the document image to be corrected, obtain the first text detection result and the first text recognition result, and calculate the first confidence level of the first text recognition result; The rotation module is used to determine the arrangement direction of the text strips in the document image to be corrected based on the first text detection result and the first text recognition result when the first confidence level is less than or equal to the first threshold, and to rotate the document image to be corrected based on the arrangement direction. The correction module is used to perform text recognition on the rotated document image to be corrected, obtain a second text recognition result, calculate a second confidence level of the second text recognition result, and correct the document image to be corrected based on the first confidence level and the second confidence level.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the document image correction method as described in any one of claims 1 to 6.

9. A computer-readable medium, characterized in that, The computer-readable medium carries computer-executable instructions, which, when executed by a processor, are used to implement the document image correction method as described in any one of claims 1 to 6.

Images