An image processing method, apparatus, electronic device, and storage medium

Abstract

The present disclosure provides an image processing method and device, electronic equipment and storage medium, the method comprises: obtaining an original image of a document; performing contour searching on the original image to obtain an initial contour set; screening the initial contour in the initial contour set to determine a target contour set; and processing the original image based on the target contour in the target contour set to obtain a target image of the document. The scheme of the present disclosure can obtain a target image corresponding to the document with cleaner content, and improve the accuracy of semantic recognition.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] In intelligent document processing, paper or electronic documents need to be parsed and their content converted into storable or usable knowledge. During this conversion process, borders within the document may lead to the generation of confusing semantic information. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] A first aspect of this disclosure provides an image processing method, the method comprising:

[0005] Get the original image of the document; Perform contour lookup on the original image to obtain an initial contour set; The initial contours in the initial contour set are filtered to determine the target contour set; Based on the target contours in the target contour set, the original image is processed to obtain the target image of the document.

[0006] In one possible implementation, the initial contours in the initial contour set are filtered to determine the target contour set, including: For each initial contour in the initial contour set, determine the bounding rectangle of the initial contour; Based on the size information of the circumscribed rectangle, the initial contour is filtered to obtain a candidate contour set; Based on the inner and outer boundary lines of the candidate contours in the candidate contour set, the candidate contours are filtered to obtain the target contour set.

[0007] In one possible implementation, the candidate contours are filtered based on the inner and outer boundary lines of the candidate contours in the candidate contour set to obtain a target contour set, including: Determine the first area enclosed by the inner boundary lines of the candidate contours in the candidate contour set, and the second area enclosed by the outer boundary lines; The first area is compared with the second area to obtain a first comparison result; If the first comparison result meets the preset area requirement, the candidate contour is taken as the target contour; Based on the target contour, a target contour set is obtained.

[0008] In one possible implementation, the initial contour is filtered based on the size information of the circumscribed rectangle to obtain a candidate contour set, including: Determine the size information of the original image; The size information of the original image is compared with the size information of the bounding rectangle to obtain a second comparison result; If the second comparison result meets the preset size requirement, the initial contour is used as the candidate contour; Based on the candidate contours, a candidate contour set is obtained.

[0009] In one possible implementation, contour lookup is performed on the original image to obtain an initial contour set, including: The original image is binarized to obtain a binarized image; Determine the gradient magnitude of each pixel in the binarized image; The gradient magnitude is compared with the gradient threshold to obtain a third comparison result; Based on the third comparison result, contour points are determined, and the contour points are connected sequentially to obtain an initial contour set.

[0010] In one possible implementation, based on the target contours in the target contour set, the original image is processed to obtain the target image of the document, including: Determine the contour mask based on the target contours in the target contour set; The target image of the document is obtained by pixel resetting the target contour in the original image based on the contour mask.

[0011] In one possible implementation, the target image of the document is input into a text recognition model to obtain the text information of the document output by the text recognition model.

[0012] A second aspect of this disclosure provides an image processing apparatus, the apparatus comprising: The image acquisition module is used to acquire the original image of the document; The contour lookup module is used to perform contour lookup on the original image to obtain an initial contour set; The contour filtering module is used to filter the initial contours in the initial contour set to determine the target contour set; The image processing module is used to process the original image based on the target contours in the target contour set to obtain the target image of the document.

[0013] A third aspect of this disclosure provides an electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0014] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0015] The image processing method, apparatus, electronic device, and storage medium disclosed herein first perform a contour search operation on the original image of the document to extract all contours in the image and obtain an initial contour set. Then, for each initial contour in the initial contour set, it is determined whether it is a target contour (i.e., a redundant border contour). Finally, based on the target contour, the original image is processed to remove redundant borders, obtaining the target image. The image processed using this method allows subsequent layout recognition models and table parsing models to avoid misidentifying redundant borders as tables, thereby achieving accurate recognition of document content and ensuring higher accuracy and reliability of the extracted structured knowledge.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0017] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0018] Figure 1 A schematic diagram of a redundant block diagram according to an embodiment of the present disclosure is shown; Figure 2 A schematic diagram illustrating the implementation flow of an image processing method according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of an inner and outer boundary line according to an embodiment of the present disclosure is shown; Figure 4 A schematic diagram illustrating the implementation flow of another image processing method according to an embodiment of the present disclosure is shown; Figure 5 A schematic diagram of an application scenario according to an embodiment of this disclosure is shown; Figure 6A schematic diagram of the composition structure of an image processing apparatus according to an embodiment of the present disclosure is shown; Figure 7 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0019] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0020] In the document processing workflow, the system first transforms the document into structured knowledge, and then uses this structured knowledge for subsequent applications. However, current document parsing methods cannot effectively distinguish between redundant borders and actual tables in a document, easily misinterpreting redundant borders as tables, leading to semantic confusion in the parsing results and affecting the accuracy and completeness of knowledge extraction. This problem further impacts the effectiveness of subsequent applications based on this structured knowledge.

[0021] In related technologies, the document page is first divided into regions using a layout recognition model. If the recognition result determines that a certain region is a table, a table parsing model is then used to extract the structure of that region. However, if the document contains... Figure 1 When redundant borders are displayed, the layout recognition model may misidentify these non-table redundant borders as table areas and send them to the table parsing model. Since redundant borders lack the row and column structure and semantic relationships of a table, the table parsing model cannot extract valid information, ultimately generating chaotic semantic data and severely affecting the overall parsing quality.

[0022] To solve the above-mentioned technical problems, this disclosure proposes the following solutions: A first aspect of this disclosure provides an image processing method, which is executed by an electronic device capable of data processing, such as... Figure 2 As shown, the method includes the following steps: Step 101: Obtain the original image of the document.

[0023] The document can be a paper document, such as a contract, report, or invoice. For such paper documents, an image acquisition device is used to photograph or scan them to obtain the corresponding original images. It can also be an electronic document, such as a PDF or Word document. For such electronic documents, page splitting and format conversion technology is used to convert the content of each page into an image format, thereby obtaining the original images corresponding to each page.

[0024] Step 102: Perform contour lookup on the original image to obtain an initial contour set.

[0025] First, the original image is binarized, transforming it into a binary image containing only the foreground and background. Then, techniques such as gradient edge detection and neighborhood connectivity search algorithms can be used to find contours, identifying all regions in the image with continuous boundary features. All the obtained independent contours are used as initial contours, and the set of all initial contours is the initial contour set.

[0026] Step 103: Filter the initial contours in the initial contour set to determine the target contour set.

[0027] The initial contours determined in step 102 include various types of contours such as redundant box contours, table contours, and contours formed by text annotation edges. However, the goal of this embodiment is to remove redundant boxes in the document, so non-redundant box contours such as table contours and text annotation contours need to be excluded, and only redundant box contours are retained as target contours. The set of all target contours is the target contour set.

[0028] Specifically, since redundant bounding box outlines typically exhibit regular rectangular structures, while non-target outlines such as image content are often irregular shapes, preliminary screening can be performed based on the shape characteristics of the initial outline. For example, by determining the number of vertices in the initial outline (e.g., retaining only closed outlines with 4 vertices), irregular outlines can be removed, leaving only rectangular outlines.

[0029] Furthermore, the dimensions of redundant boxes typically follow a fixed correspondence with the document page ratio. For example, the width of a redundant box is close to the document page width, and the ratio of its height to the page height is within a preset range. However, the dimensions of non-target outlines, such as text markup boxes, differ significantly from the page ratio. Therefore, a secondary screening can be performed based on the dimensional characteristics of the initial outlines, eliminating outlines whose dimensions do not conform to the characteristics of redundant boxes.

[0030] Finally, redundant boxes have two closed contours: an outer boundary line and an inner boundary line. The size ratio of the inner and outer boundary lines (e.g., the ratio of the inner boundary line size to the outer boundary line size is consistently within a preset range) and the hierarchical relationship (the inner boundary line is a sub-contour of the outer boundary line) have clear characteristics. In contrast, text markers and other contours typically contain only one box structure. Table box contours, due to the presence of internal dividing lines, have significantly different inner and outer boundary line size ratios compared to redundant boxes. Therefore, further screening is performed based on the structural features of the contours.

[0031] In this way, other contours that do not conform to the redundant box features can be filtered out, and only the contours that conform to the redundant box features can be retained as the target contours.

[0032] Step 104: Based on the target contours in the target contour set, process the original image to obtain the target image of the document.

[0033] Using the target contours in the target contour set as the basis for locating redundant borders, methods such as pixel value replacement and region background filling are employed to process the areas covered by the target contours in the original image, making them blend seamlessly with the document background. Specifically, the pixel values ​​of this region can be directly replaced with the pixel values ​​of the document background; or the region can be smoothly filled using neighborhood pixel interpolation, thereby completely eliminating redundant borders in the original image. This target image, free from redundant border interference, can be directly used for subsequent tasks such as structured knowledge extraction.

[0034] The method in this embodiment first performs a contour search operation on the original image of the document to extract all contours in the image and obtain an initial contour set. Then, for each initial contour in the initial contour set, it is determined whether it is a target contour (i.e., a redundant border contour). Finally, based on the target contour, the original image is processed to remove redundant borders from the image, obtaining the target image. The image processed by this method can prevent subsequent layout recognition models and table parsing models from misidentifying redundant borders as tables, thereby achieving accurate recognition of document content and ensuring that the extracted structured knowledge has higher accuracy and reliability.

[0035] In another embodiment of this disclosure, contour searching is performed on the original image to obtain an initial contour set. Specifically, this can be achieved through the following technical means: binarizing the original image to obtain a binarized image; determining the gradient magnitude of each pixel in the binarized image; comparing the gradient magnitude with a gradient threshold to obtain a third comparison result; determining contour points based on the third comparison result, and connecting the contour points sequentially to obtain the initial contour set.

[0036] First, the original image is binarized to convert it into a binary image containing only black and white pixel values. This can be achieved using the following formula (1): (1) in, The pixel value (x, y) of a pixel in a binarized image; The pixel value (x, y) of a pixel in the original image is represented by T; T represents the binarization threshold. Considering that the background of document images is generally white, T can be 254 here. That is, when the gray value of a pixel in the original image is less than 254, it is determined to be the foreground, and otherwise it is determined to be the background.

[0037] Then, gradient analysis is performed on the above binarized image to determine the gradient magnitude of each pixel. This can be achieved through the following formulas (2) and (3): , (2) (3) Where I represents the pixel grayscale value of the binarized image; The gradient component of a pixel (x, y) along the x-axis is used to characterize the intensity of grayscale change in that direction. G is the gradient component of pixel (x, y) along the y-axis, used to characterize the intensity of grayscale change in that direction; G is the gradient magnitude of pixel (x, y), characterizing the degree of grayscale abrupt change at that pixel. The larger the gradient magnitude, the more likely the pixel is located at the image edge, i.e., a contour point.

[0038] Then, the gradient magnitude is compared with the gradient threshold. Based on the third comparison result, the contour points are selected, which can be achieved using the following formula (4): (4) in, The value is the outline point marker, where 1 indicates that the pixel is an outline point and 0 indicates that it is not an outline point; For pixels (x, y) The gradient magnitude; TH is the preset gradient threshold, which can be flexibly set according to the actual application scenario. It can be a single fixed value or a range of numbers.

[0039] This step calculates the gradient magnitude of each pixel. Compare with the gradient threshold TH; if the third comparison result is... If the result of the third comparison is greater than or equal to TH, then the pixel is determined to be a contour point and marked as 1; if the result of the third comparison is... If the value is less than TH, it is determined to be a non-contour point and marked as 0.

[0040] Finally, all the selected contour points are continuously connected using a connectivity analysis algorithm to obtain the initial contour. This can be achieved using techniques such as the 8-neighborhood search algorithm and the connected component labeling algorithm. For example, it can be achieved using the following formula (5): (5) in, This represents the contour point currently being traversed; express The 8 neighboring pixels are the 8 neighboring pixels of the current pixel in the directions of top, bottom, left, right and four diagonals. This is a direction calculation function used to determine the direction of the next contour point to be traversed based on the relative position of the current contour point and its neighboring contour points; Next represents the next contour point to be traversed.

[0041] This step begins by starting with the first pixel marked as a contour point, serving as the initial traversal starting point. Then, it searches for all contour points marked as 1 within the 8-neighborhood of this starting point. The function determines the optimal connection direction and obtains the first contour point. The next contour point is then used as the new... Repeat the aforementioned steps, continuously searching for and connecting contour points within its 8-neighborhood until no new neighboring contour points are found. All consecutive contour points obtained during this traversal process form a complete initial contour. The set of all obtained initial contours is the final initial contour set.

[0042] The solution in this embodiment accurately captures all contours in the original image of the document through a standardized contour recognition process, forming an initial contour set, which provides complete contour data support for subsequent targeted screening of redundant contours.

[0043] In another embodiment of this disclosure, the initial contours in the initial contour set are filtered to determine the target contour set. This can be achieved by the following technical means: for each initial contour in the initial contour set, the circumscribed rectangle of the initial contour is determined; based on the size information of the circumscribed rectangle, the initial contours are filtered to obtain a candidate contour set; based on the inner and outer boundary lines of the candidate contours in the candidate contour set, the candidate contours are filtered to obtain the target contour set.

[0044] Since the initial contour set contains various types of contours, including redundant box contours, table box contours, text box contours, and image box contours, while the subsequent processing targets only redundant box contours, it is necessary to filter out the redundant box contours that belong to the target contours. Specifically, text box contours are first filtered and removed based on the size features of the initial contours to obtain a candidate contour set. Then, table box contours are filtered and removed based on the number and / or shape features of the inner and outer boundary lines of the candidate contours, finally obtaining the target contour set containing only redundant box contours.

[0045] Specifically, redundant bounding boxes are characterized by their size being proportionate to the document page, their overall area being relatively large, and their width approaching the width of the document page. Text boxes and image boxes, on the other hand, are generally smaller, and their aspect ratios differ significantly from the page proportions. Therefore, based on these size characteristics, text box and image box outlines can be filtered out and removed. For example, size thresholds such as aspect ratio thresholds and area thresholds that match the document page proportions can be set. For each initial outline in the initial outline set, its circumscribed rectangle is determined through methods such as outline boundary pixel extreme value extraction and minimum bounding rectangle fitting. Then, the width, height, aspect ratio, and area of ​​this circumscribed rectangle are extracted as its size information. This size information is then compared with the preset size threshold range to filter out initial outlines whose sizes conform to the characteristics of redundant bounding boxes, while removing outlines such as text boxes and image boxes that are significantly smaller or disproportionate, resulting in a candidate outline set.

[0046] Because redundant box outlines possess specific morphological characteristics in their inner and outer boundaries, such as area ratios, length-to-width ratios, and quantity, and because other types of table frames, like the inner and outer boundaries of a table frame, exhibit dimensional and quantity characteristics distinct from these features due to the presence of dividing lines, redundant box outlines, i.e., target outlines, can be selected based on the specific morphological characteristics of their inner and outer boundaries. Figure 3 As shown, the outer boundary line refers to the closed contour formed by the outer edge of the redundant frame line, and the inner boundary line refers to the closed contour formed by the inner edge of the redundant frame line. The area between the outer boundary line and the inner boundary line constitutes the frame line entity area of ​​the redundant frame. For example, by setting a threshold for the number of inner boundary lines, for each candidate contour in the candidate contour set, its outer boundary line and all its inner boundary lines are extracted. Redundant frame contours usually only have a complete closed outer boundary line, and there is only one single closed inner boundary line parallel to the outer boundary line inside. However, within the outer boundary line of a table frame contour, there are multiple independent sub-closed inner boundary lines formed by row and column dividing lines. All table frame contours with more than the threshold number of inner boundary lines are excluded, and the remaining contours are the redundant frame contours. All redundant frame contours together form the target contour set.

[0047] The solution in this embodiment first eliminates non-target contours based on the size difference between redundant boxes and other types of boxes. Then, it further eliminates non-target contours based on the differences in the inner and outer boundary features between redundant boxes and other types of boxes. This allows for rapid location and filtering of target contours, significantly improving the accuracy and reliability of redundant box contour filtering.

[0048] In another embodiment of this disclosure, the initial contour is filtered based on the size information of the circumscribed rectangle to obtain a candidate contour set. Specifically, this can be achieved through the following technical means: determining the size information of the original image; comparing the size information of the original image with the size information of the circumscribed rectangle to obtain a second comparison result; if the second comparison result meets the preset size requirements, the initial contour is used as a candidate contour; and the candidate contour set is obtained based on the candidate contours.

[0049] First, for each initial contour in the initial contour set, the size information of its bounding rectangle is determined based on the pixel coordinates of the bounding rectangle. This size information includes the width and height of the bounding rectangle. Specifically, this can be achieved through the following formulas (6)-(8): , (6) , (7) , (8) in, The x-coordinates of all pixels in the current initial outline bounding rectangle; The ordinate of all pixels in the current initial outline bounding rectangle; and These represent the minimum and maximum values ​​of the x-coordinate of the bounding rectangle pixels, respectively. and These represent the minimum and maximum values ​​of the ordinate of the bounding rectangle pixels, respectively. The width of the bounding rectangle is represented by ; the height of the bounding rectangle is represented by .

[0050] Secondly, obtain the size information of the original image, including its width and height. There is a clear quantitative relationship between the size information of the original image and the size information of the circumscribed rectangle (excluding text boxes and other non-target outlines). This relationship can be set according to the document type and general typesetting standards. For example, legal documents typically use a font size of 12pt or 14pt, and the length or width of the circumscribed rectangle formed by a single text element usually accounts for less than 0.03% of the length or width of the entire document image. Based on the general typesetting characteristics of such documents, corresponding preset size requirements can be set.

[0051] Therefore, the dimensions of the circumscribed rectangles of each initial contour can be quantitatively compared with the dimensions of the original image to obtain a second comparison result. This second comparison result can be the ratio or difference between the circumscribed rectangle and the original image in dimensions such as length and width. Then, it is determined whether the second comparison result meets the preset size requirements. The form of the preset size requirement corresponds to the type of the second comparison result. For example, if the second comparison result is a length ratio, the preset size requirement is a length ratio threshold. If the second comparison result is a width ratio, the preset size requirement is a width ratio threshold.

[0052] To illustrate with a ratio: the length of the circumscribed rectangle is compared to the length of the original image, resulting in a second comparison result, the length ratio. A preset size requirement is that the length ratio of the circumscribed rectangle relative to the original image is greater than 0.03. If this ratio is greater than a threshold, the second comparison result is deemed to meet the preset size requirement, and the initial contour is included in the candidate contours; conversely, if the ratio is less than or equal to the threshold, the initial contour is determined to be a text-type non-target contour and is discarded. After the above filtering, all retained candidate contours together constitute the candidate contour set.

[0053] The solution in this embodiment uses the original image size information as a basis to accurately distinguish between non-target contours such as text boxes and redundant box contours, and efficiently removes contours with significantly smaller size proportions to obtain a candidate contour set.

[0054] In another embodiment of this disclosure, candidate contours are filtered based on the inner and outer boundary lines of the candidate contours in the candidate contour set to obtain a target contour set. Specifically, this can be achieved through the following technical means: determining a first area enclosed by the inner boundary line of the candidate contours in the candidate contour set and a second area enclosed by the outer boundary line; comparing the first area and the second area to obtain a first comparison result; if the first comparison result meets the preset area requirement, the candidate contour is taken as the target contour; and the target contour set is obtained based on the target contours.

[0055] First, the first area enclosed by the inner boundary line of the candidate contour can be determined using Green's formula, polygon area integration, etc. Similarly, the second area enclosed by the outer boundary line can be calculated. Both can be achieved using the following formula (9): (9) Where n represents the number of pixels that make up the boundary line; A represents the coordinates of the i-th pixel on the boundary line; A represents the area of ​​the region enclosed by the boundary line.

[0056] The area ratio of the region enclosed by the outer and inner boundaries of a redundant bounding box (target contour) has specific characteristics. Non-redundant bounding boxes, such as tables, have internal row and column dividing lines, resulting in a significant difference in the area ratio of their inner and outer boundaries compared to redundant bounding boxes. Therefore, a judgment rule can be constructed based on this difference, namely a preset area requirement, to distinguish redundant bounding boxes from other non-target contours. The preset area requirement can take the form of an area ratio, area difference, etc., and its specific form corresponds to the comparison method of the first and second areas. For example, if the first and second areas are compared as a ratio, the preset area requirement is an area ratio threshold; if they are compared as a difference, the preset area requirement is an area difference threshold.

[0057] For example, the ratio of the first area of ​​the region enclosed by the inner boundary line to the second area of ​​the region enclosed by the outer boundary line is calculated to obtain the first comparison result. If the first comparison result meets the preset area requirement, the candidate contour is determined to be the target contour, which can be achieved through the following formula (10): (10) Where A represents the first area enclosed by the inner boundary line; The second area is represented by the outer boundary line; if the first area A and the second area... If the ratio is greater than 0.9, then the candidate contour is determined as the target contour. ).

[0058] All candidate contours that meet the preset area requirements are taken as target contours, and the set of all target contours is the target contour set.

[0059] This embodiment achieves precise differentiation between two types of contours by quantitatively determining the area ratio of the regions enclosed by the inner and outer boundary lines and based on the characteristic differences between redundant boxes and table boxes. This method efficiently filters out target contours through simple area calculations and ratio comparisons.

[0060] In another embodiment of this disclosure, the original image is processed based on the target contours in the target contour set to obtain the target image of the document. Specifically, this can be achieved through the following technical means: determining a contour mask based on the target contours in the target contour set; and resetting the pixels of the target contours in the original image based on the contour mask to obtain the target image of the document.

[0061] First, a contour mask is generated based on all the selected target contours. This mask is used to mark the pixel positions of all target contours in the original image. Using this contour mask, the pixels at the target contour positions in the original image are reset to pixel values ​​consistent with the document background, thereby removing the target contours (i.e., redundant boxes). At the same time, valid content such as text and tables in the original image are preserved. Specifically, this can be achieved through the following formula (11): (11) Where I(x, y) represents the pixel value of the original image at coordinates (x, y), M(x, y) represents the mask value of the contour mask at coordinates (x, y), M(x, y)=1 indicates that the coordinate belongs to the pixel region corresponding to the redundant box, and M(x, y)=0 indicates that the coordinate belongs to the valid content region in the original image other than the redundant box; W represents the pixel value of the document background; I′(x, y) represents the pixel value of the processed target image at coordinates (x, y).

[0062] The scheme in this embodiment generates a matching contour mask based on the target contour, marks the redundant box area using the contour mask, and resets the pixels in the area, thereby achieving seamless elimination of redundant boxes in the original image and finally outputting a clean target image without redundant box interference.

[0063] In another embodiment of this disclosure, the method further includes: inputting the target image of the document into a text recognition model to obtain the text information of the document output by the text recognition model.

[0064] The target image of the document is a pure image that has had redundant bounding boxes removed and retains complete and valid content. The text recognition model can be an optical character recognition model or a deep learning text recognition model adapted to the document scenario. This model can be deployed on the electronic device implementing this solution or independently on a remote server. After the target image is input into the text recognition model, the model can accurately detect and recognize the text regions in the image, and finally output high-precision text information from the document.

[0065] To better understand the above embodiments, a specific example is provided below for illustration. Figure 4 , Figure 5 As shown, this example includes the following steps: Extracting the original image from a PDF file, such as Figure 5 As shown in (a). Then, utilizing the feature that the page background is white, hard thresholding binarization is performed on the original image to generate a binarized image. A contour search operation is then performed on the binarized image to obtain all contours and form an initial contour set, as shown in (a). Figure 5 As shown in (b), traverse all initial contours in the initial contour set and determine whether they are redundant box contours, i.e., target contours.

[0066] The specific judgment logic is as follows: First, determine if it is a closed rectangle with 4 vertices. If so, calculate its bounding rectangle. Filter out outlines whose bounding rectangle length is ≥ 0.03 times the image length or whose bounding rectangle width is ≥ 0.03 times the image width. Exclude text outlines to obtain a candidate outline set. For each contour in the candidate contour set, determine whether it has an inner boundary line; if it does, calculate the first area enclosed by the inner boundary line and the second area enclosed by the outer boundary line. If the first area is greater than or equal to 0.9 times the second area, then the contour is identified as the target contour and added to the contour list. Figure 5 As shown in (c).

[0067] Next, redundant bounding box masks are generated based on the target contours in the contour list, and pixel reset operations are performed on each redundant bounding box region in the original image. The final output is the target image after removing the redundant bounding boxes, as shown below. Figure 5 As shown in (d), the target image can be input into subsequent layout recognition and table parsing models, which in turn generate structured knowledge.

[0068] A second aspect of this disclosure provides an image processing apparatus, such as... Figure 6 As shown, the device includes: Image acquisition module 501 is used to acquire the original image of the document; The contour search module 502 is used to perform contour search on the original image to obtain an initial contour set; The contour filtering module 503 is used to filter the initial contours in the initial contour set to determine the target contour set. Image processing module 504 is used to process the original image based on the target contours in the target contour set to obtain the target image of the document.

[0069] In another embodiment of this disclosure, the contour filtering module 503 is further configured to determine the circumscribed rectangle of each initial contour in the initial contour set; filter the initial contours based on the size information of the circumscribed rectangle to obtain a candidate contour set; and filter the candidate contours based on the inner and outer boundary lines of the candidate contours in the candidate contour set to obtain a target contour set.

[0070] In another embodiment of this disclosure, the contour filtering module 503 is further configured to determine a first area enclosed by the inner boundary line of the candidate contour in the candidate contour set, and a second area enclosed by the outer boundary line; compare the first area with the second area to obtain a first comparison result; if the first comparison result meets the preset area requirement, use the candidate contour as the target contour; and obtain the target contour set based on the target contour.

[0071] In another embodiment of this disclosure, the contour filtering module 503 is further configured to determine the size information of the original image; compare the size information of the original image with the size information of the bounding rectangle to obtain a second comparison result; if the second comparison result meets the preset size requirements, use the initial contour as a candidate contour; and obtain a candidate contour set based on the candidate contour.

[0072] In another embodiment of this disclosure, the contour lookup module 502 is further configured to perform binarization processing on the original image to obtain a binarized image; determine the gradient magnitude of each pixel in the binarized image; compare the gradient magnitude with a gradient threshold to obtain a third comparison result; determine contour points based on the third comparison result; and connect the contour points sequentially to obtain an initial contour set.

[0073] In another embodiment of this disclosure, the image processing module 504 is further configured to determine a contour mask based on the target contours in the target contour set; and to perform pixel resetting on the target contours in the original image based on the contour mask to obtain the target image of the document.

[0074] In another embodiment of this disclosure, the device further includes a recognition module for inputting a target image of a document into a text recognition model to obtain text information of the document output by the text recognition model.

[0075] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0076] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0077] like Figure 7As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0078] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0079] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform image processing methods by any other suitable means (e.g., by means of firmware).

[0080] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0081] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0082] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0083] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0084] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0085] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0086] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0088] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: Get the original image of the document; Perform contour lookup on the original image to obtain an initial contour set; The initial contours in the initial contour set are filtered to determine the target contour set; Based on the target contours in the target contour set, the original image is processed to obtain the target image of the document.

2. The image processing method according to claim 1, characterized in that, The initial contours in the initial contour set are filtered to determine the target contour set, including: For each initial contour in the initial contour set, determine the bounding rectangle of the initial contour; Based on the size information of the circumscribed rectangle, the initial contour is filtered to obtain a candidate contour set; Based on the inner and outer boundary lines of the candidate contours in the candidate contour set, the candidate contours are filtered to obtain the target contour set.

3. The image processing method according to claim 2, characterized in that, Based on the inner and outer boundary lines of the candidate contours in the candidate contour set, the candidate contours are filtered to obtain a target contour set, including: Determine the first area enclosed by the inner boundary lines of the candidate contours in the candidate contour set, and the second area enclosed by the outer boundary lines; The first area is compared with the second area to obtain a first comparison result; If the first comparison result meets the preset area requirement, the candidate contour is taken as the target contour; Based on the target contour, a target contour set is obtained.

4. The image processing method according to claim 2, characterized in that, Based on the size information of the circumscribed rectangle, the initial contour is filtered to obtain a candidate contour set, including: Determine the size information of the original image; The size information of the original image is compared with the size information of the bounding rectangle to obtain a second comparison result; If the second comparison result meets the preset size requirement, the initial contour is used as the candidate contour; Based on the candidate contours, a candidate contour set is obtained.

5. The image processing method according to claim 1, characterized in that, Contour lookup is performed on the original image to obtain an initial contour set, including: The original image is binarized to obtain a binarized image; Determine the gradient magnitude of each pixel in the binarized image; The gradient magnitude is compared with the gradient threshold to obtain a third comparison result; Based on the third comparison result, contour points are determined, and the contour points are connected sequentially to obtain an initial contour set.

6. The image processing method according to claim 1, characterized in that, Based on the target contours in the target contour set, the original image is processed to obtain the target image of the document, including: Determine the contour mask based on the target contours in the target contour set; The target image of the document is obtained by pixel resetting the target contour in the original image based on the contour mask.

7. The method according to claim 1, characterized in that, The method further includes: The target image of the document is input into the text recognition model to obtain the text information of the document output by the text recognition model.

8. An image processing apparatus, characterized in that, The device includes: The image acquisition module is used to acquire the original image of the document; The contour lookup module is used to perform contour lookup on the original image to obtain an initial contour set; The contour filtering module is used to filter the initial contours in the initial contour set to determine the target contour set; The image processing module is used to process the original image based on the target contours in the target contour set to obtain the target image of the document.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images