Methods, apparatus, and media for extracting target document portions from images.

Abstract

Embodiments of this disclosure relate to a method, apparatus, and medium for extracting a target document portion from an image. According to the method, an image to be processed is acquired, the image to be processed having a target document portion and a background portion; the image to be processed is binarized, and then subjected to a first filtering process and a second filtering process on the binarized image to obtain a first image and a second image, wherein the first image has less noise than the second image, and the second image has higher sharpness than the first image; based on the first image, the coordinates of four corner points of the target document portion are determined; and based on the determined coordinates of the four corner points, the target document portion is extracted from the second image. Thus, a clear and usable target document portion can be accurately and efficiently extracted from the image.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of image processing, and more specifically to a method, apparatus, and medium for extracting target document portions from an image. Background Technology

[0002] When capturing images of documents using image capture devices such as cameras, the captured images typically include both the document portion and the background portion of the surrounding environment (e.g., a desktop). These images often also contain various distortions such as shadows (e.g., shadows from the surrounding environment), distortion, blurred edges, or low lighting, significantly hindering the accurate extraction of the document portion from the image. Currently, users usually need to manually extract the desired document portion from such images, but this is inefficient. Furthermore, due to the aforementioned distortions, it is difficult for users to accurately pinpoint the precise outline of the target document within such images, making it difficult to accurately extract target documents completely devoid of background. Moreover, users typically only perform distortion correction after extracting the target document, further reducing processing efficiency.

[0003] Therefore, it is necessary to provide a technique for automatically extracting target document portions from images, so as to accurately and efficiently extract clear and usable document portions from images. Summary of the Invention

[0004] To address the aforementioned issues, this disclosure provides a method, apparatus, and medium for extracting target document portions from an image, enabling accurate and efficient extraction of clear and usable document portions from an image.

[0005] According to a first aspect of this disclosure, a method for extracting a target document portion from an image is provided, comprising: acquiring an image to be processed, the image to be processed having a target document portion and a background portion; performing binarization processing on the image to be processed, such that the binarized image to be processed is subjected to a first filtering processing and a second filtering processing respectively, to obtain a first image and a second image respectively, wherein the first image has less noise than the second image and the second image has higher sharpness than the first image; determining the coordinates of four corner points of the target document portion based on the first image; and extracting the target document portion from the second image based on the determined coordinates of the four corner points.

[0006] According to a second aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect of this disclosure.

[0007] In a third aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, characterized in that the computer instructions are used to cause the computer to operate according to a method of the first aspect of this disclosure.

[0008] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0010] Figure 1 A schematic diagram of an exemplary system 100 for implementing a method for extracting a target document portion from an image according to embodiments of the present disclosure is shown.

[0011] Figure 2 A flowchart of a method 200 for extracting a target document portion from an image according to an embodiment of the present disclosure is shown.

[0012] Figure 3 A flowchart is shown of a method 300 for determining the coordinates of four corner points of a target document portion based on a first image obtained through a first filtering process, according to an embodiment of the present disclosure.

[0013] Figure 4 A flowchart of a method 400 for extracting a target document portion from a second image obtained by a second filtering process, according to an embodiment of the present disclosure, is shown.

[0014] Figure 5 A flowchart of a method 500 for determining the coordinates of four corner points of a tilt-corrected target document portion according to an embodiment of the present disclosure is shown.

[0015] Figure 6A A schematic diagram of an exemplary image to be processed after binarization according to an embodiment of the present disclosure is shown;

[0016] Figure 6BA schematic diagram of an exemplary third image according to an embodiment of the present disclosure is shown.

[0017] Figure 6C A schematic diagram of an exemplary fourth image according to an embodiment of the present disclosure is shown.

[0018] Figure 6D A schematic diagram of the final extracted target document portion according to an embodiment of this public is shown.

[0019] Figure 7 A block diagram of an electronic device 700 according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be more thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0021] In the following description, certain specific details are set forth for the purpose of illustrating embodiments of the various inventions to provide a thorough understanding of these embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0022] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0023] Throughout this specification, references to "one embodiment" or "some embodiments" indicate that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Therefore, the appearance of "in one embodiment" or "in some embodiments" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0024] Furthermore, the terms first, second, third, fourth, etc., used in the specification and claims are used only for the purpose of clarity of description to distinguish between different objects, and do not limit the size or other order of the objects they describe.

[0025] As mentioned above, when capturing images of documents using image capture devices such as cameras, the captured images typically include both the document portion and the background portion of the surrounding environment (e.g., a desktop). These images often also contain various distortions such as shadows (e.g., shadows from the surrounding environment), distortion, blurred edges, or low lighting, significantly hindering the accurate extraction of the document portion from the image. Currently, users typically extract the desired document portion manually from such images, but this is inefficient. Furthermore, due to the aforementioned distortions, it is difficult for users to accurately pinpoint the precise outline of the target document within such images, making it challenging to accurately extract target documents completely devoid of background. Moreover, users often perform distortion processing only after extracting the target document, further reducing processing efficiency.

[0026] To at least partially address one or more of the aforementioned problems and other potential issues, exemplary embodiments of this disclosure propose a method for extracting a target document portion from an image, comprising: acquiring an image to be processed, the image to be processed having a target document portion and a background portion; binarizing the image to be processed, and then performing a first filtering process and a second filtering process on the binarized image to obtain a first image and a second image, wherein the first image has less noise than the second image, and the second image has higher sharpness than the first image; determining the coordinates of four corner points of the target document portion based on the first image; and extracting the target document portion from the second image based on the determined coordinates of the four corner points. In this manner, a clear and usable target document portion can be accurately and efficiently extracted from an image.

[0027] In the following sections, specific examples of this solution will be described in more detail with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram of an exemplary system 100 for implementing a method for extracting target document portions from an image according to embodiments of the present disclosure is shown. Figure 1As shown, system 100 includes a computing device 110, a network 120, and a server 130. The computing device 110 and server 130 can interact via the network 120 (e.g., the Internet). In this disclosure, server 130 can provide a service for providing an image to be processed to the computing device 110. The computing device 110 can communicate with server 130 via network 120 to extract a target document portion from the acquired image to be processed. The computing device 110 may include at least one processor 112 and at least one memory 114 coupled to the at least one processor 112, the memory 114 storing instructions 116 executable by the at least one processor 112, which, when executed by the at least one processor 112, perform the method 200 described below. Note that herein, the computing device 110 may be part of server 130 or may be independent of server 130. Of course, in some embodiments, server 130 and network 120 may not be included, and the user may directly load or transfer the image to be processed to computing device 110, so that computing device 110 can extract the target document portion from the image to be processed. The specific structure of computing device 110 or server 130 can be combined as follows, for example. Figure 7 As stated above.

[0029] Figure 2 A flowchart of a method 200 for extracting a target document portion from an image according to an embodiment of the present disclosure is shown. Method 200 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 7 The method is performed at the illustrated electronic device 700. It should be understood that method 200 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0030] In step 202, an image to be processed is obtained, which has a target document portion and a background portion.

[0031] In this disclosure, the image to be processed can be an image obtained by capturing a document using an image capture device such as a camera.

[0032] The image to be processed includes the captured portion of the target document and the captured background portion of the surrounding environment (e.g., the desktop) of the document.

[0033] The purpose of this disclosure is to extract information-identifiable target document portions from such images to be processed.

[0034] In some embodiments, before performing further processing on the image to be processed, it can be determined whether the image is usable, primarily by determining whether it is unusable due to excessively large shadows. If the image to be processed contains excessively large shadows (i.e., dense shadows), then after extracting the target document portion from the image, such shadows will prevent the effective recognition of text and other content information from the target document portion. In some implementations, to determine whether the image to be processed is unusable due to excessively large shadows, a grayscale histogram of the image to be processed can be generated first. Then, based on the grayscale histogram, it is determined whether the proportion of pixels with grayscale values ​​less than or equal to a predetermined value (e.g., 49) in the image to be processed is greater than or equal to a predetermined threshold (e.g., 24%). In response to determining that the proportion is greater than the predetermined threshold, the image to be processed is determined to be unusable due to excessively large shadows; otherwise, the image to be processed is determined to be usable, and subsequent processing can be performed.

[0035] In step 204, the image to be processed is binarized to obtain a binarized image (e.g., as shown in the image). Figure 6A As shown, a first filtering process and a second filtering process are performed respectively to obtain a first image and a second image. In this disclosure, the first image obtained by the first filtering process has less noise than the second image obtained by the second filtering process, but the second image has higher sharpness than the second image.

[0036] In this disclosure, binarizing the image to be processed can help improve the speed of subsequent processing.

[0037] In some embodiments, binarizing the image to be processed may include the following steps.

[0038] First, the image to be processed is converted to grayscale. In this disclosure, converting the image to grayscale will help improve the speed of subsequent processing.

[0039] Then, the grayscale image is subjected to fuzziness assessment to determine whether it is sufficiently clear. In some implementations, fuzziness assessment of the grayscale image may include multiplying the grayscale image with a pre-defined Laplacian convolution kernel to obtain the corresponding response image. In this disclosure, the pre-defined Laplacian convolution kernel may, for example, be a 3x3 convolution kernel, such as... After obtaining the response map, its variance can be calculated. If the variance is less than a predetermined threshold, the document image to be processed is determined to be blurry; otherwise, the document image to be processed is determined to be sufficiently sharp. The predetermined threshold can be, for example, the median grayscale value of all pixels in the image to be processed.

[0040] If the grayscale image is determined to be sufficiently clear through fuzzy detection, it can be binarized. For example, an adaptive thresholding algorithm can be used to achieve this binarization. Specifically, a thresholding algorithm can be used to determine the threshold value of the grayscale image. Then, the grayscale values ​​of pixels with values ​​greater than the determined threshold are set to 1, and the grayscale values ​​of pixels with values ​​less than the determined threshold are set to 0, thus achieving binarization of the grayscale image.

[0041] Currently, no filtering process can simultaneously address both image noise and sharpness. Therefore, in this disclosure, by using two different images obtained by applying different filtering processes to the same image to be processed when determining the coordinates of the four corner points of the target document portion and when extracting the target document portion, the accuracy of the determined coordinates can be effectively guaranteed, while also ensuring the clarity of the extracted target document portion. As described above, in step 204, the first image is obtained by performing a first filtering process on the binarized image to be processed, while the second image is obtained by performing a second filtering process on the binarized image to be processed. The first image has less noise than the second image, but the second image has higher sharpness. On the one hand, since the first image has less noise than the second image, recording the first image to determine the coordinates of the four corner points of the target document portion in subsequent processing helps improve the accuracy of the identified four corner points. On the other hand, since the second image has higher sharpness than the first image, the content information (such as text) contained in the second image is sharper and there are no shadows. Therefore, extracting the target document portion from the second image will help to obtain a higher quality document portion, which will make it easier to identify useful content information such as text from the obtained document image.

[0042] In some embodiments, the first filtering process can be, for example, median filtering, and the second filtering process can be, for example, bilateral filtering. Median filtering mainly involves dividing the image (e.g., the grayscale image mentioned earlier) into multiple image blocks (e.g., each image block is 5*5 pixels in size), and then replacing the pixel value of each pixel in each image block with the median pixel value of that image block, that is, the median pixel value is the median of the pixel values ​​of all pixels in the 5*5 image block. Median filtering can effectively remove most of the noise in the image to be processed, so using the first image obtained by median filtering to determine the coordinates of the four corner points of the target document portion can effectively ensure the accuracy of the determined coordinates of the four corner points. However, while removing noise, median filtering also blurs the useful content included in the image (e.g., the text in the document portion), so if the target document portion is extracted from the first image, the useful information in the obtained target document portion will not be clear enough.

[0043] Bilateral filtering can eliminate shadows in an image and sharpen the content information (e.g., text) in the image, but some other noise cannot be removed. Therefore, using a second image obtained through bilateral filtering to extract the target document portion will help improve the clarity of the useful information in the obtained target document portion.

[0044] In step 206, based on the first image, the coordinates of the four corner points of the target document portion are determined.

[0045] As mentioned earlier, since the first image contains less noise, determining the coordinates of the four corner points of the target document object based on the first image ensures the accuracy of the determined coordinates.

[0046] The following will combine Figure 3 Step 206 will be described in further detail.

[0047] In step 208, the target document portion is extracted from the second image based on the coordinates of the four corner points.

[0048] Since the second image is the same as the first image, both are obtained by processing the same image to be processed, the coordinates of the four corner points of the target document portion determined based on the first image are actually the coordinates of the four corner points of the target document portion in the second image. Therefore, based on the coordinates determined in step 206, the required target document portion can be accurately extracted from the second image.

[0049] Moreover, as mentioned earlier, since the second image has relatively higher sharpness, the target document portion extracted from the second image can be clearer compared to the first image.

[0050] The following will be based on Figure 4 Step 208 will be described in further detail.

[0051] Figure 3 A flowchart of a method 300 for determining the coordinates of four corner points of a target document portion based on a first image obtained through a first filtering process, according to an embodiment of the present disclosure, is shown. Method 300 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 7 The method is performed at the illustrated electronic device 700. It should be understood that method 300 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0052] In step 302, the tilt of the target document portion is determined based on the first image.

[0053] The first image is the same as the first image obtained in step 204.

[0054] In some embodiments, determining the tilt of a target document portion based on a first image may include detecting straight lines in the first image to determine the tilt of the target document portion by determining the tilt angle of a horizontal line among the detected lines relative to a horizontal border of the first image. In other embodiments, determining the tilt of a target document portion may include detecting straight lines in the first image to determine the tilt of the target document portion by determining the tilt angle of a vertical line among the detected lines relative to a vertical border of the first image.

[0055] In the above embodiments, checking for straight lines in the first image can be achieved, for example, using the Hough transform algorithm.

[0056] In step 304, the first image is rotated based on the tilt determined in step 302 to obtain a third image with the tilt-corrected target document portion.

[0057] After determining the tilt of the target document portion, the tilt can be corrected by rotating the first image by a corresponding angle, resulting in a third image that contains the tilt-corrected target document portion. For example, Figure 6B An example of a third image is shown. According to... Figure 6B The target document portion in the third image has been tilted and has low noise, but the text portion is not particularly clear.

[0058] In step 306, edge detection is performed on the third image to preliminarily determine the four border lines of the tilt-corrected target document portion.

[0059] In some embodiments, the Canny edge detection algorithm may be used, for example, to perform edge detection on the third image.

[0060] In this disclosure, in order to further eliminate noise and improve the accuracy of the corner coordinates of the target document portion determined in subsequent processing, the third image may be filtered again after step 306 (e.g., mean filtering, the mean filtering kernel of which may be, for example, 5*5), and then two dilation and erosion operations may be performed.

[0061] In step 308, a connected component analysis is performed on the third image to determine the coordinates of the four corner points of the tilt-corrected target document portion based on multiple connected components identified from the third image and the four preliminarily determined border lines.

[0062] In this disclosure, multiple connected regions can be identified from the third image through connected component analysis. Based on at least these connected regions, the coordinates of the four corner points of the tilt-corrected target document portion can be determined.

[0063] The following will combine Figure 5 Step 308 will be described in further detail.

[0064] By employing the above methods, this disclosure is able to accurately and efficiently determine the location of a portion of the target document.

[0065] Figure 4 A flowchart of a method 400 for extracting a target document portion from a second image obtained through a second filtering process, according to an embodiment of the present disclosure, is shown. Method 400 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 7 The method is performed at the illustrated electronic device 700. It should be understood that method 400 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0066] In step 402, the second image is rotated based on the tilt determined in step 302 to obtain a fourth image with the tilt-corrected target document portion.

[0067] The second image is the second image obtained in step 204 above.

[0068] Figure 6C An example of the fourth image is shown below. According to... Figure 6C The target document portion in the fourth image has been tilted and the text is clearer than in the third image, but the noise in the fourth image is greater than in the third image.

[0069] Since the second image underwent the same processing as the first image in step 304 in step 402, the tilt-corrected target document portion is actually located in the same position in the resulting third and fourth images. Therefore, using the coordinates of the tilt-corrected target document portion determined relative to the third image in step 308, the coordinates of the tilt-corrected target document portion in the fourth image can be determined, thereby facilitating the extraction of the tilt-corrected target document portion from the fourth image.

[0070] In step 404, based on the coordinates of the four corner points, the border of the tilt-corrected target document portion in the fourth image is determined.

[0071] Specifically, by connecting the corresponding coordinate points in the fourth image in sequence with straight lines, the border of the tilt-corrected target document portion in the fourth image can be obtained.

[0072] In step 406, using perspective transformation, based on the determined border and the border of the fourth image itself, an affine transformation is performed on the fourth image to extract an image from the fourth image that includes only the tilt-corrected target document portion, and the extracted image is enlarged to a horizontal width equal to the horizontal width of the minimum bounding rectangle of the fourth image.

[0073] In this disclosure, the image extracted from the fourth image through affine transformation is an image with the background removed, meaning that the image only includes the tilt-corrected target document portion. Furthermore, in this disclosure, through this affine transformation, the four sides of the extracted image are stretched proportionally, so that the horizontal width of the image is equal to the horizontal width of the minimum bounding rectangle of the fourth image.

[0074] For example Figure 6D An image of the final extracted target document portion is shown.

[0075] By employing the methods described above, this disclosure is able to obtain the desired clear image of the target document portion.

[0076] Figure 5 A flowchart of a method 500 for determining the coordinates of four corner points of a tilt-corrected target document portion according to an embodiment of the present disclosure is shown. Method 500 may be derived from, for example... Figure 1 The computing device 110 shown can be used for execution, and can also be used in Figure 7 The method is performed at the illustrated electronic device 700. It should be understood that method 500 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0077] In step 502, it is determined whether the ratio of the area of ​​the largest connected region among the multiple connected regions to the area of ​​the third image (i.e., the third image obtained in step 304) is greater than a predetermined first ratio.

[0078] The multiple connected regions mentioned in step 502 are the same connected regions identified by the connected region analysis of the third image mentioned in step 308. These connected regions can be sorted by their area to determine the connected region with the largest area, i.e., the largest connected region.

[0079] In this disclosure, the predetermined first proportion can be determined, for example, by performing multiple identification statistics, and can be, for example, 1 / 2 or 2 / 3, etc.

[0080] On the one hand, in step 504, if the ratio of the area of ​​the largest connected region to the area of ​​the third image is greater than the predetermined first ratio, the coordinates of the four corner points of the smallest bounding rectangle of the largest connected region are determined.

[0081] In step 506, if at least one of the four corner points of the minimum bounding rectangle of the maximum connected region is outside the maximum connected region, then the corner point is replaced by the point within the maximum connected region that is closest to the corner point.

[0082] It should be understood that if two or more of the four corner points of the minimum bounding rectangle of the maximum connected region are located outside the maximum connected region, then the operation mentioned in step 506 can be performed on each of these corner points.

[0083] In step 508, the coordinates of the four points obtained are used as the coordinates of the four corner points of the target document portion after tilt correction.

[0084] Conversely, if all four corner points of the minimum bounding rectangle of the maximum connected region are located within the maximum connected region, steps 506 and 508 are not executed, and the coordinates of the four corner points of the minimum bounding rectangle of the maximum connected region are directly used as the coordinates of the four corner points of the target document portion after tilt correction.

[0085] On the other hand, in step 510, if the ratio of the area of ​​the largest connected region among the multiple connected regions (i.e., the multiple connected regions obtained by performing connected region analysis on the third image) to the area of ​​the third image is less than a predetermined first ratio, the multiple connected regions are spliced ​​together to obtain a spliced ​​connected region.

[0086] In this disclosure, before stitching together the above multiple connected regions, the third image can be subjected to two dilation and erosion operations to further remove noise from the third image, making subsequent processing more accurate.

[0087] In some embodiments, stitching together multiple connected regions includes the following steps. First, one or more connected regions (i.e., multiple connected regions obtained by performing connected region analysis on a third image) whose horizontal width of the smallest bounding rectangle is less than a predetermined second proportion of the horizontal width of the smallest bounding rectangle of the largest connected region are identified. In this disclosure, the predetermined second proportion may be, for example, 1 / 2. Since connected regions with excessively small areas (i.e., connected regions whose horizontal width of the smallest bounding rectangle is less than a predetermined second proportion of the horizontal width of the smallest bounding rectangle of the largest connected region) may be connected regions caused by some noise in the image, which usually affects the accuracy of determining the coordinates of the target document portion, it is necessary to first identify and remove such small connected regions in this disclosure. However, there is a special case where these small connected regions may include connected regions involving titles. If connected regions involving titles are also removed, the corresponding titles will be excluded, thus making the finally extracted document portion inaccurate. It should be understood that titles are usually located at the top of the article (e.g., document title) or inside the article (e.g., chapter title). Analysis revealed that headings located within the text, such as chapter titles, are typically merged into a larger connected region along with other text. Therefore, this disclosure does not result in the erroneous removal of connected regions containing headings within the text. However, for headings located at the top of the text, such as document titles, the corresponding connected regions are usually smaller. Therefore, these headings are easily removed along with the smaller connected regions. Thus, it is necessary to determine the nature of such connected regions before removal. Since connected regions associated with headings at the top of the text are typically located in the upper half of the corresponding image, this determination is feasible. Specifically, if one of the identified connected regions (i.e., one or more connected regions whose horizontal width of the smallest bounding rectangle is less than a predetermined second proportion of the horizontal width of the smallest bounding rectangle of the largest connected region) is located in the upper half of the third image, optical character recognition (i.e., OCR recognition) is performed on that connected region to determine whether it contains a heading.

[0088] On one hand, in response to determining that the connected region contains the title, the coordinates of the top-left and bottom-right corners of the connected region are determined. Simultaneously, the coordinates of the four corners of each of the multiple connected regions (i.e., the multiple connected regions obtained through connected region analysis of the third image) need to be determined, excluding the one or more connected regions determined (i.e., one or more connected regions where the horizontal width of the previously determined minimum bounding rectangle is less than a predetermined second proportion of the horizontal width of the minimum bounding rectangle of the maximum connected region). Subsequently, the coordinates of the four corners of the spliced ​​connected region can be determined based on the coordinates of the four corners of each of the determined other connected regions, the coordinates of the top-left and bottom-right corners of the connected region containing the title, and the four preliminarily determined border lines (i.e., the four border lines determined in step 306). Specifically, the coordinates of the four points closest to the four preliminarily determined border lines among the coordinates of the four corners of each of the determined other connected regions and the coordinates of the top-left and bottom-right corners of the connected region containing the title can be determined as the coordinates of the four corners of the spliced ​​connected region. For example, the coordinates of the top and left border lines, which are closest to the four initially determined border lines among the coordinates mentioned above, can be used as the top left corner of the spliced ​​connected region. The coordinates of the top and right border lines, which are closest to the four initially determined border lines among the coordinates mentioned above, can be used as the top right corner of the spliced ​​connected region. And so on, until the coordinates of the four corner points of the spliced ​​connected region are finally determined.

[0089] On the other hand, in response to determining that the connected region does not contain a title, one or more of the determined connected regions (i.e., one or more connected regions whose horizontal width of the previously determined minimum bounding rectangle is less than a predetermined second proportion of the horizontal width of the minimum bounding rectangle of the maximum connected region) are removed. Simultaneously, the coordinates of the four corner points of the minimum bounding rectangle of each of the remaining connected regions need to be determined. Subsequently, based on the coordinates of the four corner points of the minimum bounding rectangle of each of the remaining connected regions and the initially determined four border lines (i.e., the four border lines determined in step 306), the coordinates of the four corner points of the spliced ​​connected region can be determined. Specifically, the coordinates of the four points closest to the initially determined four border lines among the coordinates of the four corner points of the minimum bounding rectangle of each of the remaining connected regions can be determined as the coordinates of the four corner points of the spliced ​​connected region. For example, the coordinates of the top and left border lines, which are closest to the four initially determined border lines among the coordinates mentioned above, can be used as the top left corner of the spliced ​​connected region. The coordinates of the top and right border lines, which are closest to the four initially determined border lines among the coordinates mentioned above, can be used as the top right corner of the spliced ​​connected region. And so on, until the coordinates of the four corner points of the spliced ​​connected region are finally determined.

[0090] In step 512, based on the spliced ​​connected regions, the coordinates of the four corner points of the tilt-corrected target document portion are determined.

[0091] The shape formed by connecting the four corner points of a spliced ​​connected region in sequence is usually a trapezoid. Therefore, to prevent the top area of ​​this trapezoid from being too small, it is necessary to determine the coordinates of its top two corner points. If the distance between these two coordinates is less than the maximum width of the connected region, then the top coordinates are replaced with the top coordinates of the maximum connected region. Specifically, if the distance between the top-left and top-right corner points of the spliced ​​connected region is less than the horizontal width of the minimum bounding rectangle of the maximum connected region, then the coordinates of the top-left and top-right corner points of the spliced ​​connected region are replaced with the coordinates of the top-left and top-right corner points of the minimum bounding rectangle of the maximum connected region, respectively, as the coordinates of the final determined top-left and top-right corner points of the target document portion.

[0092] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1 The computing device 110 shown can be implemented by an electronic device 700. As shown, the electronic device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 702 or loaded from storage unit 708 into random access memory (RAM) 703. The random access memory 703 may also store various programs and data required for the operation of the electronic device 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0093] Multiple components in electronic device 700 are connected to input / output interface 705, including: input unit 706, such as keyboard, mouse, microphone, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0094] The various processes and procedures described above, such as methods 200-500, can be executed by the central processing unit 701. For example, in some embodiments, methods 200-500 can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via read-only memory 702 and / or communication unit 709. When the computer program is loaded into random access memory 703 and executed by central processing unit 701, one or more actions of methods 200-500 described above can be performed.

[0095] This disclosure relates to methods, apparatus, systems, electronic devices, computer-readable storage media, and / or computer program products. A computer program product may include computer-readable program instructions for performing various aspects of this disclosure.

[0096] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0097] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge computing devices. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0098] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0099] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0100] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0101] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0103] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for extracting target document portions from an image, comprising: Acquire an image to be processed, the image having a target document portion and a background portion; The image to be processed is binarized so that the binarized image is subjected to a first filtering process and a second filtering process to obtain a first image and a second image, respectively. The first image has less noise than the second image, and the second image has higher sharpness than the first image. Based on the first image, determine the coordinates of the four corner points of the target document portion; as well as Based on the coordinates of the four corner points, the target document portion is extracted from the second image.

2. The method according to claim 1, wherein the first filtering process is median filtering, and the second filtering process is bilateral filtering.

3. The method according to claim 1, wherein determining the coordinates of the four corner points of the target document portion based on the first image includes: Based on the first image, determine the tilt angle of the target document portion; The first image is rotated based on the tilt angle to obtain a third image with the tilt-corrected target document portion; Edge detection is performed on the third image to preliminarily determine the four border lines of the tilt-corrected target document portion; as well as Connectivity analysis is performed on the third image to determine the coordinates of the four corner points of the tilt-corrected target document portion based on multiple connected regions identified from the third image and four preliminarily determined border lines. In response to the ratio of the area of ​​the largest connected region to the area of ​​the third image being greater than a predetermined first ratio, the coordinates of the four corner points of the minimum bounding rectangle of the largest connected region are determined; if at least one corner point of the minimum bounding rectangle of the largest connected region is outside the largest connected region, the corner point is replaced by the point closest to that corner point within the largest connected region; if all four corner points of the minimum bounding rectangle of the largest connected region are within the largest connected region, the coordinates of the four corner points of the minimum bounding rectangle are used as the coordinates of the four corner points of the tilt-corrected target document portion; the coordinates of the four points obtained are then used as the coordinates of the four corner points of the tilt-corrected target document portion; or In response to the fact that the ratio of the area of ​​the largest connected region among the plurality of connected regions to the area of ​​the third image is less than the predetermined first ratio, the plurality of connected regions are stitched together to obtain a stitched connected region; based on the stitched connected region, the coordinates of the four corner points of the tilt-corrected target document portion are determined.

4. The method according to claim 3, wherein extracting the target document portion from the second image based on the determined coordinates of the four corner points comprises: The second image is rotated based on the tilt angle to obtain a fourth image with the tilt-corrected target document portion; Based on the coordinates of the four corner points, the border of the tilt-corrected target document portion in the fourth image is determined; as well as Using perspective transformation, based on the determined border and the border of the fourth image itself, an affine transformation is performed on the fourth image to extract an image from the fourth image that includes only the tilt-corrected portion of the target document, and the extracted image is enlarged to a size whose horizontal width is equal to the horizontal width of the minimum bounding rectangle of the fourth image.

5. The method of claim 3, wherein determining the tilt of the target document portion based on the first image comprises: Straight lines are detected in the first image so that the tilt of the target document portion can be determined by determining the tilt angle of the horizontal straight lines among the detected straight lines relative to the horizontal border of the first image. or Straight lines are detected in the first image so that the tilt of the target document portion can be determined by determining the tilt angle of the vertical straight line among the detected straight lines relative to the vertical border of the first image.

6. The method according to claim 3, wherein splicing the plurality of connected regions comprises: Identify one or more connected regions in which the horizontal width of the smallest bounding rectangle of the plurality of connected regions is less than a predetermined second ratio of the horizontal width of the smallest bounding rectangle of the largest connected region; If one of the identified connected regions is located in the upper half of the third image, optical character recognition is performed on the connected region to determine whether the connected region contains a title. In response to the determination that the connected region contains a header, the coordinates of the top-left and bottom-right corners of the connected region are determined. Determine the coordinates of the four corner points of each of the multiple connected regions other than the one or more determined connected regions; Based on the coordinates of the four corner points of each other connected region, the coordinates of the top left and bottom right corner points of the connected region containing the title, and the four border lines initially determined, the coordinates of the four corner points of the spliced ​​connected region are determined.

7. The method according to claim 6, further comprising determining the coordinates of the four corner points of the tilt-corrected target document portion based on a plurality of connected regions determined from the third image and four preliminarily determined border lines, including: In response to determining that the connected region does not contain a header, one or more of the determined connected regions are removed; Determine the coordinates of the four corner points of the minimum bounding rectangle of each of the remaining connected regions; Based on the coordinates of the four corner points of the minimum bounding rectangle of each remaining connected region and the four preliminarily determined border lines, the coordinates of the four corner points of the spliced ​​connected region are determined.

8. The method of claim 3, wherein determining the coordinates of the four corner points of the tilt-corrected target document portion based on the spliced ​​connected regions comprises: If the distance between the top-left and top-right corners of the four corners of the spliced ​​connected region is less than the horizontal width of the minimum bounding rectangle of the largest connected region, then the coordinates of the top-left and top-right corners of the four corners of the spliced ​​connected region are replaced with the coordinates of the top-left and top-right corners of the minimum bounding rectangle of the largest connected region as the coordinates of the top-left and top-right corners of the final determined target document portion.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

Images