Document image correction method and apparatus, electronic device, and readable medium

Abstract

The application discloses a document image correction method and device, electronic equipment and readable medium. The document image correction method comprises the following steps: obtaining the offset probability of a to-be-corrected document image relative to a standard direction, which is offset by 0 degrees, 90 degrees, 180 degrees and 270 degrees, respectively; when the maximum probability in the offset probability is greater than a first threshold, correcting the to-be-corrected document image to the standard direction based on the offset angle corresponding to the maximum probability; when the maximum probability in the offset probability is less than or equal to the first threshold, performing text detection and text recognition on the to-be-corrected document image, judging whether the text recognition result has semantic information, and correcting the to-be-corrected document image to the standard direction based on the judgment result. The document image correction method and device provided by the application can correct the to-be-corrected document image based on the offset probability of the to-be-corrected document image, and can correct the to-be-corrected document image to the standard direction, so that the OCR recognition accuracy of the document image is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a document image correction method, apparatus, electronic device, and readable medium. Background Technology

[0002] In existing technologies, text content in document images is generally recognized using OCR (Optical Character Recognition) technology. During OCR recognition, the orientation of the document image must match the OCR recognition direction; otherwise, errors in the OCR recognition results are likely to occur, resulting in a large amount of garbled text.

[0003] Therefore, in view of the above-mentioned technical problems, it is necessary to provide a document image correction method, apparatus, electronic device and readable medium. Summary of the Invention

[0004] The purpose of this invention is to provide a document image correction method, apparatus, electronic device, and readable medium, which can correct the orientation of document images to improve the OCR recognition accuracy of document images.

[0005] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0006] In a first aspect, the present invention provides a document image correction method, comprising:

[0007] Obtain the offset probabilities of the document image to be corrected relative to its standard direction at 0°, 90°, 180°, and 270°, respectively; when the maximum probability among the offset probabilities is greater than a first threshold, correct the document image to be corrected to the standard direction based on the offset angle corresponding to the maximum probability; when the maximum probability among the offset probabilities is less than or equal to the first threshold, identify the text content of the document image to be corrected, determine whether the identification result of the text content has semantic information, and correct the document image to be corrected to the standard direction based on the semantic information of the identification result.

[0008] In one or more embodiments, correcting the document image to be corrected based on the semantic information of the recognition result includes: when the recognition result has semantic information, determining that the current orientation of the document image to be corrected is the standard orientation.

[0009] In one or more embodiments, recognizing the text content of the document image to be corrected, determining whether the recognition result of the text content has semantic information, and correcting the document image to be corrected to the standard direction based on the semantic information of the recognition result, includes: performing text detection on the document image to be corrected, performing text recognition on the text detection result to obtain the recognition result of the text content of the document image to be corrected; when the recognition result does not have semantic information, determining the offset angle of the document image to be corrected based on a text detection model with horizontal and vertical text tag detection functions, and correcting the document image to be corrected to the standard direction based on the offset angle.

[0010] In one or more embodiments, recognizing the text content of the document image to be corrected, determining whether the recognition result of the text content has semantic information, and correcting the document image to be corrected to the standard direction based on the semantic information of the recognition result, includes: performing text detection on the document image to be corrected, performing text recognition on the text detection result to obtain the recognition result of the text content of the document image to be corrected; when the recognition result does not have semantic information, determining the offset angle of the document image to be corrected based on the results of the text detection and the text recognition, and correcting the document image to be corrected to the standard direction based on the offset angle.

[0011] In one or more embodiments, determining the offset angle of the document image to be corrected based on the results of the text detection and the text recognition includes: performing text detection on the document image to be corrected based on a lateral text detection model, performing text recognition on the results of the text detection, and determining the offset angle of the text strips in the document image to be corrected based on the results of the text recognition.

[0012] In one or more embodiments, determining the offset angle of the text bars in the document image to be corrected based on the text recognition result includes: determining the proportion of single-character text boxes relative to the total text boxes in the text detection result based on the text recognition result; when the proportion is greater than a second threshold, the offset angle of the text bars in the document image to be corrected is determined to be 90° or 270°; when the proportion is less than or equal to the second threshold, the offset angle of the text bars in the document image to be corrected is determined to be 180°.

[0013] In one or more embodiments, correcting the document image to be corrected based on the offset angle includes: when the offset angle of the document image to be corrected is 180°, rotating the document image to be corrected by 180° to correct the document image to be corrected to the standard direction.

[0014] In one or more embodiments, correcting the document image to be corrected based on the offset angle includes: when the offset angle of the document image to be corrected is 90° or 270°, rotating the document image to be corrected by 90°; recognizing the text content of the rotated document image to be corrected, and determining whether the recognition result of the text content has semantic information; when the recognition result has semantic information, determining that the current direction of the rotated document image to be corrected is the standard direction; when the recognition result does not have semantic information, rotating the rotated document image to be corrected by 180° to correct the document image to be corrected to the standard direction.

[0015] In one or more embodiments, obtaining the offset probabilities of a document image to be corrected being offset by 0°, 90°, 180°, and 270° relative to its standard orientation includes: inputting the document image to be corrected into a rotation model; classifying the image features of the document image to be corrected based on the rotation model; and outputting the offset probabilities of the document image to be corrected being offset by 0°, 90°, 180°, and 270° relative to its standard orientation based on the classification results.

[0016] In a second aspect, the present invention provides a document image correction device, comprising:

[0017] An acquisition module is used to acquire the offset probabilities of the document image to be corrected relative to its standard direction at 0°, 90°, 180°, and 270°, respectively; a first correction module is used to correct the document image to be corrected to the standard direction based on the offset angle corresponding to the maximum probability when the maximum probability in the offset probabilities is greater than a first threshold; a second correction module is used to recognize the text content of the document image to be corrected when the maximum probability in the offset probabilities is less than or equal to the first threshold, determine whether the recognition result of the text content has semantic information, and correct the document image to be corrected to the standard direction based on the semantic information of the recognition result.

[0018] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the document image correction method as described above.

[0019] Fourthly, the present invention provides a computer-readable medium carrying computer-executable instructions, which, when executed by a processor, are used to implement the document image correction method as described above.

[0020] Compared with existing technologies, the document image correction method and apparatus provided by this invention obtains the offset probabilities of the document image to be corrected relative to its standard orientation at 0°, 90°, 180° and 270° respectively, and corrects the document image to be corrected based on the offset probabilities. This can correct the document image to its standard orientation, thereby improving the OCR recognition accuracy of the document image. The method can directly perform rotation correction on document images with high confidence of offset angle based on the offset probabilities, and perform text recognition and semantic classification on document images with low confidence of offset angle. At the same time, it can further determine the offset angle of the document image by combining the recognition results, thereby improving the correction accuracy of the document image. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of the document image correction method according to one embodiment of the present invention;

[0022] Figure 2 This is a flowchart of a document image correction method according to one embodiment of the present invention;

[0023] Figure 3 This is a structural block diagram of a document image correction device according to an embodiment of the present invention;

[0024] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0026] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0027] To facilitate understanding of the technical solutions of this application, the technical terms that may appear in this invention will be explained in detail below.

[0028] Document images: These are images containing text content within a document. For example, they can be images in PDF or other formats created by photographing or scanning printed text. In a PDF document, each page can be considered a separate document image.

[0029] Standard orientation: This refers to the orientation that matches the recognition system's recognition direction. Typically, inputting document images into the recognition system with a standard orientation yields higher recognition accuracy. Generally, document images with a standard orientation are more in line with human reading habits.

[0030] OCR (Optical Character Recognition) refers to the process of using electronic devices to examine printed characters on paper, determining their shape by detecting dark and light patterns, and then translating the shape into computer text using character recognition methods. In other words, for printed characters, it uses optical methods to convert the text in paper documents into black and white dot matrix document images, and then uses computers to process and analyze the images to convert the characters or text in the images into editable text.

[0031] OCR systems need to convert characters or text in the text area of ​​a document image into editable text. The recognition orientation of the document image directly affects the accuracy and efficiency of the OCR system. If the recognition orientation of the document image is incorrect, the OCR system may recognize the characters or text in an incorrect form, or fail to recognize them at all. For example, in a Chinese OCR system, if the image recognition orientation is incorrect, the text may be recognized as traditional Chinese characters or garbled text.

[0032] To avoid the aforementioned problems, the document image correction method and apparatus provided by the present invention can correct document images so that they have the correct recognition direction.

[0033] Please refer to Figure 1 The diagram shown illustrates an exemplary application scenario of the document image correction method provided by this invention. Figure 1 The illustrated implementation scenario includes a client 101, a document image correction server 102, and a network 103. Network 103 is the medium used to provide a communication link between the client 101 and the document image correction server 102. Network 103 can include various connection types, such as wired or wireless communication links or fiber optic cables, and can be at least one of a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN).

[0034] Client 101 can be an electronic device for providing images of the document to be corrected. For example, the electronic device can be a mobile terminal such as a smartphone, tablet, or laptop, or a terminal such as a desktop computer or a projector computer; this embodiment of the invention does not limit the type of device.

[0035] Document image correction server 102 refers to a server used to run any document image correction program and provide corresponding document image correction services. Document image correction server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0036] Users can upload the document image to be corrected to the document image correction server 102 via client 101. The document image correction server 102 performs image feature classification on the document image to obtain the offset probabilities of the document image to be corrected relative to its standard direction at 0°, 90°, 180°, and 270°, respectively. The document image is then corrected based on these offset probabilities. When the maximum probability among the offset probabilities is greater than a first threshold, the document image correction server 102 can correct the document image to the standard direction based on the offset angle corresponding to the maximum probability. When the maximum probability among the offset probabilities is less than or equal to the first threshold, the document image correction server 102 can recognize the text content of the document image to be corrected, determine whether the recognition result of the text content has semantic information, and correct the document image to the standard direction based on the semantic information of the recognition result.

[0037] Please refer to Figure 2 The diagram shown is a flowchart of a document image correction method according to an embodiment of the present invention. This document image correction method specifically includes the following steps:

[0038] S201: Obtain the offset probabilities of the document image to be corrected relative to its standard orientation, which are 0°, 90°, 180° and 270° respectively.

[0039] It should be noted that in this embodiment, the offset angle refers to the angle of clockwise deflection relative to the standard direction. Of course, in other embodiments, the offset angle can also be the angle of counterclockwise deflection relative to the standard direction.

[0040] Understandably, document images (pages) in documents (such as PDF documents) typically include four orientations: 0° (same as the standard orientation), 90° (90° off from the standard orientation), 180° (180° off from the standard orientation), and 270° (270° off from the standard orientation). Among these, the 0° orientation is the standard orientation for matching OCR recognition, and the OCR recognition results obtained under this orientation are more accurate.

[0041] In one exemplary embodiment, the method of obtaining the offset probabilities of the document image to be corrected relative to its standard direction at 0°, 90°, 180° and 270° respectively includes: inputting the document image to be corrected into a rotation model, classifying the image features of the document image to be corrected based on the rotation model; and outputting the offset probabilities of the document image to be corrected relative to its standard direction at 0°, 90°, 180° and 270° based on the classification result.

[0042] It should be noted that the image features of the document image to be corrected can be extracted through artificial intelligence models (such as neural network models), or manually by staff using image feature extraction software. Extracted image features include texture features, color features, spatial relationship features, etc.

[0043] The rotation model can be trained using an existing four-class classification model. The rotation model outputs four directional categories: 0°, 90°, 180°, and 270°, and also outputs the probability that the document image to be corrected belongs to each directional category. The rotation model can be a multi-layer neural network model or a traditional classifier, such as a Support Vector Machine (SVM) or Random Forest.

[0044] For a multi-layer neural network rotation model, it can include a shallow convolutional network and a self-attention neural network connected in sequence. The shallow convolutional network can be used to extract image feature data of the document image to be corrected. The self-attention neural network can be used to classify the document image to be corrected based on the image feature data, with the document image to be corrected being biased by 0°, 90°, 180°, and 270° relative to its standard orientation, respectively. Optionally, the shallow convolutional network can include multiple convolutional blocks and a fully connected layer, each convolutional block including a convolutional (Conv) layer, a pooling (Pool) layer, a batch normalization (BN) layer, and a ReLU activation (ReLU) layer.

[0045] S202: When the maximum probability in the bias probabilities is greater than the first threshold, the document image to be corrected is corrected to the standard direction based on the bias angle corresponding to the maximum probability.

[0046] It should be noted that the first threshold is used to measure the reliability of the bias probability results output by the rotation model. Generally, if the maximum probability in the bias probability output by the rotation model is greater than this first threshold, the maximum probability of the output can be considered reliable. The first threshold can be set according to actual needs; usually, different types of models, different training methods, and different training data may all lead to different first thresholds.

[0047] For example, in one embodiment, the first threshold can be set to 80%. Assuming the rotation model outputs bias probabilities of 0°, 90°, 180°, and 270° for a document image to be corrected, these probabilities are 6%, 2%, 90%, and 2%, respectively. The maximum probability among these bias probabilities is 90%, which corresponds to a bias angle of 180°. Since this maximum probability (90%) is greater than the first threshold (80%), the document image to be corrected can be considered to be biased by 180° relative to its standard orientation. In this case, the rotation model can rotate the document image to be corrected by 180° based on its current bias angle (180°), thus correcting the document image to the standard orientation.

[0048] It is understandable that when the offset angle of the document image to be corrected is 0°, it means that the current orientation of the document image is the standard orientation, and no further rotation correction is needed. When the offset angle of the document image to be corrected is 180°, rotating the document image 180° clockwise or counterclockwise will correct it to the standard orientation. When the offset angle of the document image to be corrected is 90°, rotating it 90° counterclockwise or 270° clockwise will correct it to the standard orientation. When the offset angle of the document image to be corrected is 270°, rotating it 270° counterclockwise or 90° clockwise will correct it to the standard orientation.

[0049] S203: When the maximum probability in the bias probability is less than or equal to the first threshold, the text content of the document image to be corrected is identified, it is determined whether the identification result of the text content has semantic information, and the document image to be corrected is corrected to the standard direction based on the semantic information of the identification result.

[0050] It is understandable that when the maximum probability in the bias probabilities is less than or equal to the first threshold, the bias probability result output by the rotation model can be considered unreliable. In this case, it is necessary to further recognize the text content of the document image to be corrected, determine whether the recognition result of the text content has semantic information, and correct the document image to be corrected to the standard orientation based on the semantic information of the recognition result.

[0051] In one exemplary embodiment, the method for recognizing the text content of the document image to be corrected specifically includes: performing text detection on the document image to be corrected, and performing text recognition on the result of the text detection to obtain the recognition result of the text content of the document image to be corrected.

[0052] It's important to note that text detection in the document image to be corrected refers to detecting text regions within the document image and separating these text regions from the background, preparing for subsequent text recognition. Text detection typically involves the following steps: image preprocessing, which involves denoising, grayscale conversion, and binarization of the input image to improve text region detection; and text line detection, which utilizes image processing and computer vision techniques to detect and segment the text in the image to form text boxes. Common text line detection algorithms include edge detection, region segmentation, and template matching-based methods. Deep neural network models can be used to implement text detection, such as CTPN (Connectionist Text Proposal Network), PSENet (Shape Robust Text Detection with Progressive Scale Expansion Network), and DBNet (Differentiable Binarization Network).

[0053] Text recognition refers to the process of segmenting and recognizing characters in text detected by text detection, converting the characters in a text image into editable text. Text recognition typically includes the following steps: character segmentation, which separates the characters from the text; character recognition, which utilizes computer vision, machine learning, and deep learning technologies to extract and classify features for higher accuracy and efficiency; and post-processing, which involves processing the recognition results to achieve more accurate text recognition. Post-processing methods include error correction, spell checking, and result correction. Neural network models, such as CRNN (Convolutional Recurrent Neural Network) and RARE (Robusttext recognizer with Automatic Rectification), can be used to implement text recognition.

[0054] In one exemplary embodiment, the method of correcting the document image to be corrected based on the semantic information judgment result of the recognition result specifically includes: when the recognition result has semantic information, determining that the current direction of the document image to be corrected is the standard direction.

[0055] It should be noted that semantic information refers to the knowledge and information people gain about things, concepts, relationships, etc., through their understanding of linguistic symbols. In natural language processing and computer science, semantic information usually refers to the meaningful information in natural language data such as text and speech; it is a high-level, abstract form of knowledge representation.

[0056] When text recognition results possess semantic information, it indicates that the context or a character within the recognized text is related to its surrounding characters. Typically, text detection and recognition of document images with a standard orientation yields text content with semantic information; however, text detection and recognition of document images with other offset orientations often result in text content lacking semantic information, including unrelated or nonsensical characters. Therefore, the presence or absence of semantic information in the text recognition results can be used to determine whether the current orientation of the document image is its standard orientation.

[0057] In one exemplary embodiment, the method of correcting the document image to be corrected based on the semantic information judgment result of the recognition result specifically includes: when the recognition result does not have semantic information, determining the offset angle of the document image to be corrected based on the results of the text detection and the text recognition, and correcting the document image to be corrected to the standard direction based on the offset angle.

[0058] Specifically, the method for determining the offset angle of the document image to be corrected based on the results of the text detection and the text recognition includes: performing text detection on the document image to be corrected based on a lateral text detection model, and performing text recognition on the results of the text detection; and determining the offset angle of the text bars in the document image to be corrected based on the results of the text recognition. Further, the method for determining the offset angle of the text bars in the document image to be corrected based on the results of the text recognition includes: determining the proportion of single-character text boxes relative to the total text boxes in the results of the text detection based on the results of the text recognition; when the proportion is greater than a second threshold, the offset angle of the text bars in the document image to be corrected is determined to be 90° or 270°; when the proportion is less than or equal to the second threshold, the offset angle of the text bars in the document image to be corrected is determined to be 180°.

[0059] It should be noted that in this embodiment, the horizontal text detection model used for text detection is trained using horizontal (i.e., text lines arranged horizontally) text samples, and the second threshold is a standard threshold used to determine the offset angle of the text entries. When the offset angle of the document image is 90° or 270°, the text detection results mostly contain single-character text boxes. When the offset angle of the document image is 0° or 180°, the text detection results contain fewer single-character text boxes. Therefore, the offset angle of the text entries in the document image can be determined based on the proportion of single-character text boxes relative to the total number of text boxes in the text detection results.

[0060] In this embodiment, the horizontal direction refers to the direction parallel to the text layout direction in the document image when the offset angle of the document image is 0° or 180°; the vertical direction refers to the direction parallel to the text layout direction in the document image when the offset angle of the document image is 90° or 270°.

[0061] Understandably, when the text content recognition result lacks semantic information under the current orientation of the document image to be corrected, it can be considered that the current orientation of the document image to be corrected is not its standard orientation (offset 0°). In this case, the current orientation of the document image to be corrected can have three offsets: 90°, 180°, and 270°. Therefore, when the proportion of single-character text boxes in the text detection result is less than or equal to the second threshold, the offset angle of the text is 180°; when the proportion of single-character text boxes in the text detection result is less than or equal to the second threshold, the offset angle of the text is 90° or 270°.

[0062] Of course, in other implementations, a text detection model with both horizontal and vertical text bar detection capabilities can be directly used to detect the arrangement direction of the text bars in the document image to be corrected, and then directly output the offset angle of the text bars based on their arrangement direction. This type of text detection model typically requires training with both horizontal and vertical text samples.

[0063] In one exemplary embodiment, the method of correcting the document image to be corrected based on the offset angle specifically includes: when the offset angle of the document image to be corrected is 180°, rotating the document image to be corrected by 180° to correct the document image to be corrected to the standard direction.

[0064] It is understood that when the offset angle of the document image to be corrected is 180°, the document image can be corrected to its standard orientation regardless of whether it is rotated 180° clockwise or counterclockwise. Therefore, when designing an automatic correction mechanism, any rotation direction (clockwise or counterclockwise) can be selected for rotational correction of the document image.

[0065] In an exemplary embodiment, the method of correcting the document image to be corrected based on the offset angle specifically includes: when the offset angle of the document image to be corrected is 90° or 270°, rotating the document image to be corrected by 90°; recognizing the text content of the rotated document image to be corrected, and determining whether the recognition result of the text content has semantic information; when the recognition result has semantic information, determining that the current direction of the rotated document image to be corrected is the standard direction; when the text recognition result does not have semantic information, rotating the rotated document image to be corrected by 180° to correct the document image to be corrected to the standard direction.

[0066] Understandably, when the offset angle of the document image to be corrected is 90° or 270°, after rotating it 90°, the offset angle may be 0° or 180°. In this case, it is necessary to recognize the text content of the rotated document image and determine whether the recognition result contains semantic information. If the recognition result contains semantic information, it can be determined that the current offset angle of the rotated document image is 0°, meaning the current direction of the rotated document image is its standard direction. If the recognition result does not contain semantic information, it can be determined that the current offset angle of the rotated document image is 180°. In this case, rotating the rotated document image another 180° will correct it to its standard direction.

[0067] In summary, the document image correction method provided by this invention obtains the offset probabilities of the document image to be corrected relative to its standard orientation at 0°, 90°, 180°, and 270°, respectively, and corrects the document image to be corrected based on these offset probabilities. This method can correct the document image to its standard orientation, thereby improving the OCR recognition accuracy of the document image. Based on the offset probabilities, this method can directly perform rotation correction on document images with high confidence in the offset angle, and perform text recognition and semantic classification on document images with low confidence in the offset angle. Furthermore, by combining the recognition results with further determination of the offset angle of the document image, the correction accuracy of the document image can be improved.

[0068] Please refer to Figure 3 As shown, based on the same inventive concept as the aforementioned document image correction method, one embodiment of the present invention provides a document image correction device 300, which includes an acquisition module 301, a first correction module 302, and a second correction module 303.

[0069] The acquisition module 301 is used to acquire the offset probabilities of the document image to be corrected relative to its standard direction at 0°, 90°, 180°, and 270°, respectively. The first correction module 302 is used to correct the document image to be corrected to the standard direction based on the offset angle corresponding to the maximum probability when the maximum probability among the offset probabilities is greater than a first threshold. The second correction module 303 is used to recognize the text content of the document image to be corrected when the maximum probability among the offset probabilities is less than or equal to the first threshold, determine whether the recognition result of the text content has semantic information, and correct the document image to be corrected to the standard direction based on the semantic information of the recognition result.

[0070] Specifically, the acquisition module 301 can be used to input the document image to be corrected into the rotation model, classify the image features of the document image to be corrected based on the rotation model, and output the offset probabilities of the document image to be corrected relative to its standard direction of 0°, 90°, 180° and 270° based on the classification results.

[0071] Specifically, recognizing the text content of the document image to be corrected includes: performing text detection on the document image to be corrected, and performing text recognition on the text detection results to obtain the recognition result of the text content of the document image to be corrected.

[0072] The second correction module 303 can be used to determine the current orientation of the document image to be corrected as the standard orientation when the recognition result has semantic information. It can also be used to determine the offset angle of the document image to be corrected based on the results of the text detection and the text recognition when the recognition result does not have semantic information, and to correct the document image to be corrected to the standard orientation based on the offset angle.

[0073] Please refer to Figure 4 As shown, embodiments of the present invention also provide an electronic device 400, which includes at least one processor 401, a memory 402 (e.g., non-volatile memory), a main memory 403, and a communication interface 404, wherein the at least one processor 401, the memory 402, the main memory 403, and the communication interface 404 are connected together via a bus 405. The at least one processor 401 is configured to invoke at least one program instruction stored or encoded in the memory 402, causing the at least one processor 401 to perform various operations and functions of the document image correction methods described in the various embodiments of this specification.

[0074] In the embodiments of this specification, electronic device 400 may include, but is not limited to: personal computer, server computer, workstation, desktop computer, laptop computer, notebook computer, mobile electronic device, smartphone, tablet computer, cellular phone, personal digital assistant (PDA), handheld device, messaging device, wearable electronic device, consumer electronic device, etc.

[0075] This invention also provides a computer-readable medium carrying computer-executable instructions, which, when executed by a processor, can be used to implement various operations and functions of the document image correction methods described in the various embodiments of this specification.

[0076] The computer-readable medium in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0077] In this invention, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0078] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, systems, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A document image correction method, characterized in that, include: Obtain the offset probabilities of the document image to be corrected relative to its standard orientation at 0°, 90°, 180° and 270° respectively; When the maximum probability in the bias probabilities is greater than the first threshold, the document image to be corrected is corrected to the standard direction based on the bias angle corresponding to the maximum probability; When the maximum probability in the bias probability is less than or equal to the first threshold, the text content of the document image to be corrected is identified, it is determined whether the identification result of the text content has semantic information, and the document image to be corrected is corrected to the standard direction based on the semantic information of the identification result. When the recognition result has semantic information, the current orientation of the document image to be corrected is determined to be the standard orientation; When the recognition result does not have semantic information, text detection is performed on the document image to be corrected based on the lateral text detection model, and text recognition is performed on the text detection result; based on the text recognition result, the proportion of single-character text boxes relative to the total text boxes in the text detection result is determined; The offset angle of the document image to be corrected is determined based on the proportion, and the document image to be corrected is corrected to the standard direction based on the offset angle; Wherein, when the proportion is greater than the second threshold, the offset angle of the text in the document image to be corrected is determined to be 90° or 270°; when the proportion is less than or equal to the second threshold, the offset angle of the text in the document image to be corrected is determined to be 180°.

2. The document image correction method as described in claim 1, characterized in that, The text content of the document image to be corrected is recognized, and it is determined whether the recognition result of the text content has semantic information. Based on the semantic information of the recognition result, the document image to be corrected is then corrected to the standard orientation, including: Text detection is performed on the document image to be corrected, and text recognition is performed on the text detection results to obtain the recognition result of the text content of the document image to be corrected; When the recognition result does not have semantic information, the offset angle of the document image to be corrected is determined based on a text detection model with horizontal and vertical text tag detection functions, and the document image to be corrected is corrected to the standard direction based on the offset angle.

3. The document image correction method as described in claim 1 or 2, characterized in that, Correcting the document image to be corrected based on the bias angle includes: When the offset angle of the document image to be corrected is 180°, the document image to be corrected is rotated by 180° to correct the document image to the standard direction.

4. The document image correction method as described in claim 1 or 2, characterized in that, Correcting the document image to be corrected based on the bias angle includes: When the offset angle of the document image to be corrected is 90° or 270°, rotate the document image to be corrected by 90°. The text content of the rotated document image to be corrected is identified, and it is determined whether the identification result of the text content has semantic information. When the recognition result has semantic information, the current orientation of the rotated document image to be corrected is determined to be the standard orientation; When the recognition result does not have semantic information, the rotated document image to be corrected is rotated 180° to correct the document image to be corrected to the standard orientation.

5. The document image correction method as described in claim 1, characterized in that, Obtain the offset probabilities of the document image to be corrected relative to its standard orientation at 0°, 90°, 180°, and 270°, respectively, including: The image of the document to be corrected is input into the rotation model, and the image features of the image to be corrected are classified based on the rotation model; Based on the classification results, the offset probabilities of the document image to be corrected relative to its standard orientation are output as 0°, 90°, 180° and 270°, respectively.

6. A document image correction device applied to the document image correction method of claim 1, characterized in that, include: The acquisition module is used to acquire the offset probabilities of the document image to be corrected relative to its standard orientation, which are 0°, 90°, 180° and 270° respectively. The first correction module is used to correct the document image to be corrected to the standard direction based on the offset angle corresponding to the maximum probability when the maximum probability in the offset probabilities is greater than the first threshold. The second correction module is used to identify the text content of the document image to be corrected when the maximum probability in the bias probability is less than or equal to the first threshold, determine whether the identification result of the text content has semantic information, and correct the document image to be corrected to the standard direction based on the semantic information of the identification result.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the document image correction method as described in any one of claims 1 to 5.

8. A computer-readable medium, characterized in that, The computer-readable medium carries computer-executable instructions, which, when executed by a processor, are used to implement the document image correction method as described in any one of claims 1 to 5.

Images