Hole filling method based on run-length encoding and image processing device

Abstract

The application discloses a hole filling method based on run-length encoding and an image processing device, and relates to the technical field of image processing.The method comprises the following steps: performing run-length encoding on a binary image; performing foreground representation, circumscribed rectangle operation, background difference set, hole elimination and filling on the run-length encoding set; and performing hole filling on the foreground target region formed by the run-length encoding instead of performing hole filling on the binary image, so that the image data is effectively compressed, the whole hole filling process is performed on the run-length encoding set, storage and processing of a large original pixel matrix are avoided, resource utilization and processing speed are improved, and the requirement of industrial high efficiency under the condition of a high-resolution image or a complex target is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a hole filling method based on run-length encoding and an image processing device. BACKGROUND

[0002] Filling holes in the interior of a target in an image is a common application that can make the segmented target more complete. Hole filling has applications in many fields. In the automation industry, it can fill holes in product surface defect images, such as metal part cracks and plastic product bubbles. In medical image processing, it can fill holes in organs or tissues caused by imaging quality or segmentation errors, such as tumor volume measurement. In the printing industry, it can fill holes in text caused by uneven lighting to improve the OCR (Optical Character Recognition) recognition rate.

[0003] In a binary image, a hole is defined as a foreground region with a pixel value of 255 completely containing a background region with a pixel value of 0. The common processing method for filling holes is to directly process the binary image. In the context of continuous industrial efficiency and technological updates, this processing method is inefficient when dealing with high-resolution images or complex targets. SUMMARY

[0004] The main purpose of the present application is to provide a hole filling method based on run-length encoding and an image processing device, aiming to solve the technical problem of low calculation efficiency and slow processing speed of hole filling algorithms in existing hole filling methods when processing high-resolution images or targets with complex contours.

[0005] To achieve the above purpose, the present application provides a hole filling method based on run-length encoding, comprising:

[0006] Obtaining a binary image, performing run-length encoding on the binary image to obtain a foreground region;

[0007] Calculating the circumscribed rectangle of the foreground region, expanding the boundary of the circumscribed rectangle to the surrounding to obtain an expanded circumscribed rectangle and a rectangular region;

[0008] Performing a difference set operation on the foreground region using the rectangular region to obtain a background region;

[0009] In the background region, remove the runs connected to the boundary of the expanded circumscribed rectangle, and remove all runs connected to these runs through connected component analysis to obtain a hole region;

[0010] Performing a union operation on the foreground region and the hole region to obtain a foreground region after hole filling.

[0011] In one embodiment, the steps of obtaining a binary image, run-length encoding the binary image to obtain a run set, and defining the run set as a foreground region include:

[0012] obtaining a binary image, run-length encoding the binary image to obtain a run set, and

[0013] defining the run set as a foreground region.

[0014] In one embodiment, the steps of calculating a bounding rectangle of the foreground region, and expanding the boundary of the bounding rectangle to obtain an expanded bounding rectangle and a rectangular region include:

[0015] calculating a bounding rectangle of the foreground region;

[0016] expanding the boundary of the bounding rectangle by one pixel to obtain an expanded bounding rectangle;

[0017] generating a rectangular region based on the expanded bounding rectangle.

[0018] In one embodiment, the bounding rectangle of the foreground region is a minimum bounding rectangle of the foreground region.

[0019] In one embodiment, the rectangular region includes the foreground region, a first region covered by the bounding rectangle but not included in the foreground region, and a second region expanded from the bounding rectangle.

[0020] In one embodiment, the steps of performing a set difference operation on the foreground region using the rectangular region to obtain a background region include:

[0021] traversing each row of runs of the rectangular region;

[0022] performing a set difference operation on the current row of runs of the rectangular region and the same row of runs in the foreground region to obtain a set difference result of the current row;

[0023] performing a set difference operation between the rectangular region and the foreground region based on the set difference result of each row to obtain a background region.

[0024] In one embodiment, the steps of performing a set difference operation on the current row of runs of the rectangular region and the same row of runs in the foreground region to obtain a set difference result of the current row include:

[0025] performing a set difference operation on the current row of runs of the rectangular region and the same row of runs in the foreground region;

[0026] when the foreground region runs are on the left / right side of the rectangular region runs without overlapping, keeping the rectangular region runs;

[0027] when the foreground region run is partially contained in the rectangular region run, the non-overlapping part of the rectangular region run and the foreground region run is reserved, and the part is divided into a front segment and a rear segment by the foreground region run;

[0028] when the foreground region run is partially contained in the rectangular region run, the non-overlapping part of the rectangular region run and the foreground region run is reserved, and the part is divided into a front segment and a rear segment by the foreground region run;

[0029] when the foreground region run is completely contained in the rectangular region run, the rectangular region run is discarded.

[0030] In an embodiment, the specific step of removing, in the background region, the run connected with the boundary of the extended bounding rectangle and removing all the runs connected with the runs through the connected component analysis to obtain the hole region comprises:

[0031] removing, in the background region, the run connected with the boundary of the extended bounding rectangle;

[0032] taking the run connected with the boundary as a seed run, marking all the runs connected with the seed run in the background region as a boundary connected region through 4-connected component growth;

[0033] removing all the runs in the boundary connected region to obtain the hole region.

[0034] In an embodiment, the run connected with the boundary of the extended bounding rectangle refers to a run satisfying one of the following conditions:

[0035] the row number of the run is equal to the row in which the top edge or the bottom edge of the extended bounding rectangle is located, and the column coordinate is between the left and right boundaries of the extended bounding rectangle;

[0036] the column number of the run is equal to the column in which the left edge or the right edge of the extended bounding rectangle is located, and the row coordinate is between the upper and lower boundaries of the extended bounding rectangle.

[0037] In addition, to achieve the above object, the application further provides an image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the hole filling method based on run coding.

[0038] The one or more technical solutions provided by the application have at least the following technical effects:

[0039] The application fills holes in the foreground target region formed by run coding instead of filling holes in a binary image to effectively compress image data, and the entire hole filling process is performed on the run coding set, thereby avoiding storage and processing of a large original pixel matrix, improving resource utilization and processing speed, and meeting the requirements of industrial efficiency under high resolution image or complex target conditions. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, those skilled in the art can obtain other drawings from these drawings without any creative effort.

[0042] Figure 1 A flowchart provided by the hole filling method based on run-length encoding of the present application;

[0043] Figure 2 A run-length encoding diagram of a binary image for an embodiment of the hole filling method based on run-length encoding of the present application;

[0044] Figure 3 A rectangular region coverage diagram provided by an embodiment of the hole filling method based on run-length encoding of the present application;

[0045] Figure 4 A same-row two-run position relationship diagram provided by an embodiment of the hole filling method based on run-length encoding of the present application;

[0046] Figure 5 A region difference set diagram provided by an embodiment of the hole filling method based on run-length encoding of the present application;

[0047] Figure 6 A background region diagram provided by an embodiment of the hole filling method based on run-length encoding of the present application.

[0048] The purpose implementation, functional features and advantages of the present application will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0050] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail with reference to the drawings and the specific embodiments of the present application.

[0051] The main solution of the embodiments of the present application is:

[0052] The present application provides a hole filling method based on run-length encoding, as shown in Figure 1 , comprising:

[0053] S100: acquire a binary image, run-length encode the binary image to obtain a foreground region;

[0054] S200: calculate a circumscribed rectangle of the foreground region, and expand the boundary of the circumscribed rectangle to the periphery to obtain an expanded circumscribed rectangle and a rectangular region;

[0055] S300: perform a difference set operation on the foreground region using the rectangular region to obtain a background region;

[0056] S400: in the background region, remove the run-lengths connected to the boundary of the expanded circumscribed rectangle, and remove all run-lengths connected to the run-lengths through connected component analysis to obtain a hole region;

[0057] S500: perform a union set operation on the foreground region and the hole region to obtain a foreground region after hole filling.

[0058] More specifically, the target hole filling in image processing refers to filling in the regions inside the target object that are completely surrounded by the foreground, should belong to the target, but present background values, so as to restore them to foreground pixel values, thereby improving the integrity and continuity of the target segmentation result. Such processing plays a key role in multiple fields: for example, in industrial automation, it can effectively fill in the holes of product surface defect images to provide a more accurate basis for quality evaluation. In the field of medical image analysis, it can be used to repair internal gaps of organs or tissues caused by imaging noise or segmentation algorithm errors, such as accurate measurement of tumor volume, to ensure the reliability of morphological analysis; in printed matter detection and OCR (Optical Character Recognition), it can compensate for broken or internally missing text strokes caused by uneven lighting or printing defects, significantly improving the accuracy of character recognition. Under the definition of a standard binary image, a hole specifically refers to a region whose pixel value is 0, and its periphery must be completely enclosed by pixels with a pixel value of 255. The current mainstream hole filling algorithm is almost based on direct operation on binary images.

[0059] However, as the requirements of industrial applications for efficiency and accuracy continue to increase, especially in the processing of high-resolution images or extremely complex target structures such as scenes containing a large number of small branches, deep and narrow gaps, or nested holes, the computational efficiency of such traditional methods based on binary image pixel-by-pixel traversal or region growing often appears to be insufficient, becoming a bottleneck restricting processing speed and real-time performance.

[0060] To address this challenge, it is necessary to seek better solutions. Better improvement ideas usually need to start from the optimization of the algorithm itself and the use of computing resources: on the one hand, explore the algorithm strategy that can reduce redundant calculation and more intelligently locate and fill holes, such as using advanced image morphology operations for efficient labeling, or designing a fast screening mechanism based on connected domain analysis; on the other hand, actively use the powerful computing power of parallel computing architecture, parallelize the original serial filling process, or combine the multi-scale processing idea, first locate the hole area on the low resolution image, and then fill the corresponding position of the original image, so as to greatly improve the processing efficiency while ensuring the filling effect. Therefore, the core of improving the hole filling technology lies in developing more intelligent and efficient algorithm model, and fully utilizing the acceleration potential of modern computing hardware.

[0061] To solve the above problems, the application provides a hole filling method based on run-length encoding, which is as follows:

[0062] In step S100, a binary image is obtained, and the binary image is run-length encoded to obtain a foreground region. This step is the data preprocessing basis of the whole algorithm, and the core goal is to convert the pixel matrix of the original binary image into a more compact and efficient data structure to facilitate subsequent operations.

[0063] The algorithm starts with inputting a standard binary image, in which it is usually agreed that the pixel value 255 represents the foreground and the pixel value 0 represents the background. Run-length encoding (RLE) is applied to this image. The core of this encoding method is to identify and record the continuous and same pixel value segments in each row of the image, which is called "run".

[0064] Specifically, the obtained binary image is run-length encoded into a run-length encoding set. The algorithm focuses on identifying and encoding the segments with pixel value 255, i.e. foreground runs. Each foreground run is usually composed of three key information: its starting coordinate position and its continuous extension length in the horizontal direction. Finally, the set of all independently encoded foreground runs constitutes a complete and non-redundant digital representation of all foreground regions in the image, i.e. the run-length encoding set defines the foreground region. This representation avoids directly processing the huge pixel matrix and greatly compresses the data amount.

[0065] In step S200, the bounding rectangle of the foreground region is calculated, and the boundary of the bounding rectangle is expanded to the four directions to obtain an expanded bounding rectangle and a rectangular region. This step aims to define an accurate and safe working range for subsequent hole positioning and background differentiation, avoiding unnecessary global processing of the whole image.

[0066] Based on the set of all foreground runs obtained from S100, the algorithm first computes the minimum Axis-Aligned Bounding Box (AABB) that can exactly enclose all foreground pixels. This box is defined by its minimum row number, maximum row number, minimum column number and maximum column number in the image coordinate system. To ensure the robustness of the operation, especially to prevent the background regions located at the edge of the original bounding box (which might be connected to the image boundary and should not be considered as holes) from being mistakenly included in the subsequent analysis, the algorithm deliberately expands the minimum bounding box by a certain number of pixels in all four directions (up, down, left, right) to form a larger expanded bounding box. The image region covered by the expanded bounding box defines a critical rectangular region. This rectangular region is the main stage for the background analysis and hole extraction in the subsequent steps (S300 and S400), significantly reducing the processing scope.

[0067] In step S300, the foreground region is subtracted from the rectangular region to obtain the background region. Within the defined rectangular region, all non-foreground pixel sets are preliminarily separated, preparing for the next step of accurately identifying the real internal holes. The core operation of this step is set difference. The algorithm regards the rectangular region defined in S200 as a complete pixel set. Then, it "subtracts" the foreground region accurately described by the run encoding in S100 from this rectangular region. The result of the set difference operation is the set of all pixel points that do not belong to the foreground target within the local range framed by the expanded bounding box, i.e., the background region. It should be noted that the background region obtained at this moment contains two types: one is the hole background located inside the foreground target and completely surrounded by the foreground; the other is the background located outside the foreground target but connected to the boundary of the expanded rectangle or possibly connected to the boundary through a connected path. Distinguishing between the two is the key task of the next step.

[0068] Step S200 accurately limits the scope of subsequent hole search and background analysis by calculating and expanding the bounding box of the foreground region. This rectangular region is usually much smaller than the entire image size. Step S300 performs a set difference operation between the foreground and the rectangular region within this local region to obtain the local background region. This localized processing strategy is the key to efficiency improvement, as it completely avoids any calculations in meaningless regions far from the target, fully concentrates computing resources on the target object and its adjacent regions, and greatly reduces invalid operations.

[0069] In step S400, in the background region, the strokes connected to the extended bounding rectangle boundary are removed, and all strokes connected to these strokes through connected component analysis are removed, to obtain a hole region. This is the core step of the algorithm, which is responsible for accurately screening the real internal holes from the preliminary extracted background region, and excluding all background regions connected to the image outside or the extended boundary.

[0070] This step operates on the background region in the rectangle region obtained in S300, which is also represented by stroke coding. First, the algorithm identifies and removes all background strokes that directly contact the extended bounding rectangle boundary defined in S200. Because these strokes are obviously connected to the space outside the rectangle region, they belong to the external background rather than the internal hole. However, simply removing the directly connected strokes is not enough, because there may be other background strokes that are not directly connected to the boundary but are connected to these boundary-connected strokes through a connected path of background pixels. In order to completely exclude all background connected to the outside, the algorithm needs to perform connected component analysis. It treats the background strokes as graph nodes and establishes a connected relationship based on spatial adjacency. The algorithm traces all other background strokes that belong to the same connected component as the boundary-connected strokes removed in the first step, and removes them together.

[0071] Finally, after two rounds of removal (direct boundary-connected strokes + all background strokes connected to them), the remaining background stroke set in the rectangle region represents those truly hole regions that are completely surrounded by the foreground region and have no connected path to any boundary. These holes are completely enclosed inside the foreground.

[0072] This step removes all background strokes that directly contact the extended bounding rectangle boundary in the local background region, which preliminarily excludes the external background. Then, through connected component analysis based on strokes, the algorithm can intelligently trace and remove all other background strokes that are connected to these boundary strokes, i.e., belong to the same connected component. This two-step strategy strictly follows the mathematical definition of holes, ensuring that only those background regions that are truly isolated inside the foreground are retained. The robustness of this method lies in its ability to effectively handle complex shapes, nested holes, and winding but connected background paths between holes and boundaries, greatly reducing the risk of false filling or missing filling.

[0073] In step S500, a set operation is performed on the foreground region and the hole region to obtain a foreground region after hole filling. The final step realizes the physical filling of holes, incorporates the identified internal hole region into the foreground, and outputs a complete target image.

[0074] The algorithm performs a union operation on the original foreground region in S100 and the accurately identified hole region in S400, which are both background run-length encoding sets, but represent the set of holes that need to be filled. Since the pixel value of the hole region in the original image is 0, and the pixel value of the foreground region is 255, performing the union operation essentially means modifying the value of all the pixels covered by the hole region from 0 to 255. After this operation, the background holes originally present inside the foreground target are "filled" into foreground pixels. Finally, the algorithm outputs the result of this union operation, that is, a new foreground region after hole filling that contains the original foreground and all the filled holes, thereby achieving the purpose of improving the completeness of the target.

[0075] The present application achieves data compression and efficient operation through run-length encoding, eliminates redundancy through bounding rectangle expansion and localized processing to accurately focus the calculation range, and ensures the accuracy and robustness of hole recognition through boundary contact rejection and run-level connected component analysis. These mechanisms work together to significantly improve processing efficiency while maintaining high filling accuracy, reduce resource consumption, and are particularly suitable for processing large-scale or complex-structure binary images, effectively solving the efficiency bottleneck problem of traditional pixel-by-pixel methods.

[0076] In an embodiment, the specific steps of obtaining a binary image, performing run-length encoding on the binary image to obtain a foreground region include:

[0077] Obtaining a binary image, performing run-length encoding on the binary image to obtain a run-length set; and defining the run-length set as a foreground region. It can be understood that the present embodiment first binarizes the collected image to obtain a binary image, represents the adjacent continuous pixels with a value of 255 in the binary image in the form of a run-length encoding to obtain a run-length encoding set, and defines the run-length encoding set as a region. The run is defined as , and the region is defined as , where row is the row coordinate of the continuous pixels with a value of 255 in a certain row of the binary image, cb is the starting column coordinate of the adjacent continuous pixels with a value of 255, and ce is the ending column coordinate of the adjacent continuous pixels with a value of 255. Therefore, the continuous pixels with a value of 255 in a certain row of the binary image can be encoded into a run. The binary image is encoded by run-length in each row and column, and finally forms a run-length set, i.e., a region . The region is defined as a foreground region. As shown in the 0th row of the binary image in Figure 2 , the pixel value of the white square is 0, and the pixel value of the gray square is 255. Performing row scanning and run-length encoding on the binary image successively obtains two runs runLength0={row = 0,cb = 1,ce = 3}, runLength1={row = 0,cb = 5,ce = 8}.

[0078] In an embodiment, the specific steps of calculating the bounding rectangle of the foreground region, extending the boundary of the bounding rectangle to the periphery to obtain an extended bounding rectangle and generating a rectangular region include:

[0079] S210: calculating the bounding rectangle of the range covered by the foreground region;

[0080] S220: extending the boundary of the bounding rectangle to the periphery by one pixel to obtain an extended bounding rectangle;

[0081] S230: generating a rectangular region based on the extended bounding rectangle.

[0082] In step S210, the bounding rectangle of the range covered by the foreground region is calculated. The bounding rectangle of the range covered by the foreground region region converted in S100 is calculated to obtain , where ptx, pty are the coordinates of the upper left corner of the bounding rectangle, width, height are the width and height of the bounding rectangle.

[0083] In step S220, the boundary of the bounding rectangle is extended to the periphery by one pixel to obtain an extended bounding rectangle. The bounding rectangle is extended to the periphery by one pixel to obtain a new bounding rectangle , where ptx’ = ptx - 1, pty’ = pty - 1, width’ = width + 2, height’ = height + 2.

[0084] In step S230, a rectangular region is generated based on the extended bounding rectangle. The bounding rectangle rect’ generated in S220 is used to generate a bounding rectangular region . This rectRegion completely fills the range covered by rect’. Therefore, rectRegion covers 3 parts of content, which are the foreground region region, the background region covered by rect and the background region added by extending rect to the periphery by one pixel. As Figure 3 shown, the content covered by the rectangular region rectRegion is composed of 3 parts, a foreground region composed of gray squares, a black solid line surrounding the gray squares; b the background region covered by rect, composed of white squares surrounded by a black solid line, i.e. the hole region holeRegion; c the background region added by extending rect to the periphery by one pixel, composed of white dashed squares. It can be seen that in the range covered by rectRegion, the background region can be divided into two types of background regions b and c. The background region is composed of b and c.

[0085] In an embodiment, the circumscribed rectangle covered by the foreground region is a minimum circumscribed rectangle of the foreground region. Using the minimum circumscribed rectangle as the basis means that the operation range of the initial delineation is absolutely accurate and has no redundancy, and it contains all the foreground pixels exactly without more or less. This is the optimal starting point for the subsequent effective boundary expansion and localization processing steps S300 and S400. Any circumscribed rectangle larger than it will introduce unnecessary additional areas at the initial stage, increasing the amount of calculation of the subsequent difference set operation and connected component analysis. The region range that needs to be processed by the subsequent core operation can be minimized to the greatest extent, laying the most powerful foundation for overall efficiency improvement. This is the key prerequisite for realizing efficient localization processing.

[0086] In addition, on the basis of the minimum circumscribed rectangle, expansion can most economically and most safely create the required boundary buffer. The purpose of expansion is purely to provide a reliable "external" reference boundary for subsequent connectivity analysis. The amount of expansion only needs to ensure that it can cover the possible recesses or gaps of the foreground object edge that need to be identified as holes, which is usually a small fixed value or an empirical value related to image resolution / foreground size. Because the basis rectangle is the smallest, the overall size of the rectangle region after expansion is also as small as possible, avoiding unnecessary computational overhead caused by excessive expansion. While providing the necessary robustness to prevent edge hole misjudgment, the total area of the rectangle region after expansion is maximized to further optimize computational efficiency.

[0087] In step S400, it is necessary to determine whether the background run is connected to the "external". The boundary of the expanded circumscribed rectangle is clearly defined as the proxy boundary of the "external". The minimum circumscribed rectangle ensures that the foreground object itself is compactly contained in the expanded region. As long as the background run touches this expanded boundary, or its connected component can connect to the background run that touches the boundary, it is considered to be connected to the "external" and should be excluded. The background region located inside the minimum circumscribed rectangle and not connected to the expanded boundary is reliably identified as a true internal hole. The combination of the minimum circumscribed rectangle and the expansion operation provides a clear, consistent, and spatially compact "external" boundary definition for connected component analysis, which is an important link to ensure the accuracy of hole identification.

[0088] In an embodiment, the rectangular region includes the foreground region, a first region covered by the circumscribed rectangle but not included in the foreground region, and a second region expanded from the circumscribed rectangle to the four sides.

[0089] In this embodiment, the rectangular region obtained in step S200 is explicitly defined to contain the following three mutually exclusive sub-regions: 1) the foreground region, i.e. the set of all foreground pixels obtained in step S100 by run-length encoding; 2) the first region, i.e. the set of all pixels inside the original minimum bounding rectangle but not belonging to the foreground region. This part of the region is essentially the background inside the original minimum bounding rectangle. It contains: the real holes inside the foreground object, i.e. the background pixels between the foreground object edge and the boundary of its minimum bounding rectangle; 3) the second region, i.e. the newly added pixel region by extending the minimum bounding rectangle to the outside. It is the annular region between the extended bounding rectangle and the original minimum bounding rectangle. This part of the region does not exist before the extension and is completely a newly added background region, serving as a boundary buffer for connectivity analysis. Therefore, the rectangular region = foreground region + background inside the minimum rectangle + extension buffer.

[0090] In an embodiment, the specific step of obtaining the background region by using the rectangular region to perform a set difference operation on the foreground region includes:

[0091] S310: traversing each row of the rectangular region;

[0092] S320: performing a set difference operation on the current row of the rectangular region and the same row of the foreground region to obtain a set difference result of the current row;

[0093] S330: based on the set difference result of each row, calculating the set difference between the rectangular region and the foreground region to obtain the background region.

[0094] It can be understood that the background region is obtained by using the rectangular region rectRegion to perform a set difference operation on the foreground region region where "-" represents a set difference operation, and the set difference can be represented as the set difference of all runs it contains. Wherein, the run difference is which describes the range covered by run a but not covered by run b, i.e. excluding the range covered by b in the range covered by a. According to the set difference between runs, the set difference between two regions can be calculated.

[0095] In step S310, each row of the rectangular region is traversed. This step is the organizational framework and iterative driver of the entire background region calculation. Its core task is to sequentially access and prepare all run-length encoding data in the rectangular region by row, laying the foundation for subsequent fine set difference operation by row.

[0096] In step S320, the current row run of the rectangular region is set difference with the run of the same row in the foreground region to obtain the set difference result of the current row. This step is the core calculation unit of S300. It accurately calculates those continuous pixel segments (i.e. background pixel segments) belonging to the rectangular region but not belonging to the foreground region within the same row. This is the key operation to realize the localized background extraction.

[0097] In step S330, based on the set difference result of each row, the set difference between the rectangular region and the foreground region is calculated to obtain the background region. This step is the result integration stage of S300. It merges the background run set difference results calculated row by row in S320 and scattered in each row to form a complete run coding set representing all background pixels in the entire rectangular region - the background region.

[0098] In an embodiment, the specific steps of calculating the set difference between the current row run of the rectangular region and the run of the same row in the foreground region to obtain the set difference result of the current row include:

[0099] The set difference between the current row run of the rectangular region and the run of the same row in the foreground region is calculated; when the foreground region run is on the left / right side of the rectangular region run without overlapping, the rectangular region run is retained; when the foreground region run partially overlaps the left / right side of the rectangular region run, the non-overlapping part of the rectangular region run is retained; when the foreground region run is completely contained in the rectangular region run, the non-overlapping part of the rectangular region run and the foreground region run is retained, and this part is divided into a front segment and a rear segment by the foreground region run; when the foreground region run completely contains the rectangular region run, the rectangular region run is discarded.

[0100] The background region is obtained by performing set difference between the rectangular region rectRegion and the foreground region region , where "-" represents the set difference operation of regions, and the set difference of regions can be represented as the set difference of all runs contained therein. Wherein, the run set difference is , which describes the range covered by run a but not covered by run b, i.e. excluding the range covered by b in the range covered by a.

[0101] Specifically, if the two runs are not in the same row, i.e. , then . If the two runs are in the same row, i.e., the positional relationship of the two runs is divided into six cases, such as Figure 4 , a, b, c, d, e, f in Figure 4 , such as a in , the run runLengtha is completely on the left side of the run runLengthb without intersection, i.e. , the set difference is Figure 4 .b, runLengtha is completely disjoint to the right of runLengthb, i.e. The difference set is As in c, runLengtha is to the left of runLengthb and partially intersects, i.e. Figure 4 The difference set is As in d, runLengtha is to the right of runLengthb and partially intersects, i.e. The difference set is Figure 4 As in e, runLengtha completely contains runLengthb, i.e. The difference set is As in f, runLengtha is completely contained in runLengthb, i.e. Figure 4 The difference set is As in f, runLengtha is completely contained in runLengthb, i.e. The difference set is As in f, runLengtha is completely contained in runLengthb, i.e. Figure 4 The difference set is As in f, runLengtha is completely contained in runLengthb, i.e.

[0102] The difference set between two regions can then be calculated according to the difference sets between the runs, as shown in Figure 5 It can be seen from Figure 5 that the result of the difference set between rectRegion and region is the background region backGroundRegion, which consists of two parts, the hole region holeRegion and the runs connected to the boundary of rect' and the 4-connected domain runs formed by the runs, as shown in Figure 6 The run set consisting of light gray squares is the hole region holeRegion, denoted as d, and the run set consisting of dark gray squares is the runs connected to the boundary of rect' and the 4-connected domain runs formed by the runs, denoted as e.

[0103] In an embodiment, the specific steps of obtaining the hole region by removing the runs connected to the boundary of the extended bounding rectangle and removing all the runs connected to the runs through connected component analysis in the background region include:

[0104] S410: removing the runs connected to the boundary of the extended bounding rectangle in the background region;

[0105] S420: taking the runs connected to the boundary as seed runs, and marking all the runs connected to the seed runs in the background region as boundary connected regions through 4-connected domain growth;

[0106] S430: removing all the runs in the boundary connected regions to obtain the hole region.

[0107] In step S410, based on the accurate geometric condition, all background strokes directly contacting the "external agent boundary" are removed to obtain a seed set. This defines the starting point of the "external". In step S420, starting from the seed stroke, all strokes connected with the seed in the remaining background strokes are found by using the 4-connected domain growth algorithm. In image processing, the neighborhood criterion of 4-connectivity is a rule for defining the spatial adjacency relationship between pixels (or image basic units). The core is that a central unit (pixel or stroke) only forms a connected relationship with its four directly adjacent units above, below, left and right. This identifies all background regions connected with the "external" in step S430. The boundary connected region is removed from the entire background region, and the remaining isolated background region is the hole.

[0108] Specifically, the background region backGroundRegion is composed of two background regions d and e, so the remaining hole region holeRegion after removing the e region is obtained. It can be seen from Figure 6 that the background region has the 4-connected characteristic. Therefore, the strokes connected with the boundary of the circumscribed rectangle rect' in the background region backGroundRegion and all 4-connected domain strokes formed by the strokes are removed, and the remaining region is the hole region holeRegion.

[0109] In an embodiment, the strokes connected with the boundary of the extended circumscribed rectangle refer to strokes satisfying one of the following conditions:

[0110] The row number of the stroke is equal to the row where the top edge or the bottom edge of the extended circumscribed rectangle is located, and the column coordinate is between the left and right boundaries of the extended circumscribed rectangle. It can be understood that this condition is specifically used to identify the background strokes located on the upper boundary and the lower boundary of the extended circumscribed rectangle. These strokes are the most direct and explicit part of the "external" agent boundary. The column number of the stroke is equal to the column where the left edge or the right edge of the extended circumscribed rectangle is located, and the row coordinate is between the upper and lower boundaries of the extended circumscribed rectangle. It can be understood that this condition is complementary to condition one and is specifically used to identify the background strokes located on the left boundary and the right boundary of the extended circumscribed rectangle, so as to ensure that the boundary contact on all four edges of the rectangle can be detected.

[0111] The two judgment conditions together constitute a complete rule for identifying all background strokes directly physically contacting the four boundaries of the extended circumscribed rectangle. They provide an explicit "seed" set for step S400.

[0112] In combination with the above embodiments, the specific steps of the hole filling method based on stroke coding are as follows:

[0113] Firstly, the collected image is binarized to obtain a binary image, and the adjacent continuous pixels with a value of 255 in the binary image are represented in a run-length encoding manner to obtain a run-length encoding set, which is defined as a region. The run is defined as , and the region is defined as , wherein row is the row coordinate of the continuous pixels with a value of 255 in a row of the binary image, cb is the starting column coordinate of the adjacent continuous pixels with a value of 255, and ce is the ending column coordinate of the adjacent continuous pixels with a value of 255. Therefore, the continuous pixels with a value of 255 in a row of the binary image can be encoded into a run. The binary image is run-length encoded row by row and column by column to finally form a run set, that is, a region . The region is defined as a foreground region in the article. As shown in the 0th row of the binary image in Figure 2 , wherein the pixel value of the white square is 0, and the pixel value of the gray square is 255, the binary image is row scanned and run-length encoded to obtain two runs runLength0={row = 0,cb = 1,ce = 3},runLength1={row = 0,cb = 5,ce = 8} in turn.

[0114] The circumscribed rectangle of the range covered by the foreground region region converted from the binary image is calculated to obtain , wherein ptx and pty are the upper left corner coordinates of the circumscribed rectangle, and width and height are the width and height of the circumscribed rectangle. The circumscribed rectangle is expanded by one pixel around the circumscribed rectangle to obtain a new circumscribed rectangle , wherein ptx’= ptx - 1,pty’= pty - 1,width’= width + 2,height’= height + 2.

[0115] Based on the expanded circumscribed rectangle rect’, a circumscribed rectangular region rectRegion is generated . This rectRegion completely fills the range covered by rect’. Therefore, rectRegion covers three parts, which are the foreground region region, the background region covered by rect, and the background region additionally covered by the expansion of rect by one pixel around the circumscribed rectangle. As shown in Figure 3 , the content covered by the rectangular region rectRegion is shown in the figure, which is composed of three parts, a foreground region composed of gray squares surrounded by a black solid line; b a background region covered by rect, which is composed of white squares surrounded by a black solid line, that is, a hole region holeRegion; and c a background region additionally covered by the expansion of rect by one pixel around the circumscribed rectangle, which is composed of white dotted squares. It can be seen that in the range covered by rectRegion, the background region can be divided into two kinds of background regions b and c. The background region is , c.

[0116] Subtract the foreground region region from the rectangular region rectRegion to get the background region , where "-" denotes the region difference operation, and the region difference can be represented as the difference of all runs it contains. The difference of runs is , which describes the range covered by run a but not covered by run b, i.e. the range covered by run a excluding the range covered by run b.

[0117] Specifically, if the two runs are not in the same row, i.e. , then . If the two runs are in the same row, i.e., the position relationship of the two runs can be divided into six cases, as shown in Figure 4 , where a, b, c, d, e, f. For example, as shown in Figure 4 , run runLengtha is completely disjointed to the left of run runLengthb, i.e. , and the difference set is . For example, as shown in Figure 4 , run runLengtha is completely disjointed to the right of run runLengthb, i.e. , and the difference set is . For example, as shown in Figure 4 , run runLengtha is partially intersected to the left of run runLengthb, i.e. , and the difference set is . For example, as shown in Figure 4 , run runLengtha is partially intersected to the right of run runLengthb, i.e. , and the difference set is . For example, as shown in Figure 4 , run runLengtha completely contains run runLengthb, i.e. , and two difference sets are generated , . For example, as shown in Figure 4 , run runLengtha is completely contained in run runLengthb, i.e. , and no difference set is generated.

[0118] Then, according to the difference set between the runs, the difference set between the two regions can be calculated, as shown in Figure 5 . From Figure 5It can be seen that the result of the difference set of rectRegion and region is the background region backGroundRegion, which consists of two parts, the hole region holeRegion and the run connected with the boundary of rect' and the 4-connected domain run formed by the run, as shown in Figure 6 The run set composed of light gray squares is the hole region holeRegion, denoted as d, and the run set composed of dark gray squares is the run connected with the boundary of rect' and the 4-connected domain run formed by the run, denoted as e.

[0119] The background region backGroundRegion consists of two background regions d and e, so the remaining region after removing the e region is the hole region holeRegion. It can be seen from Figure 6 The background region is of 4-connected characteristic. Therefore, after removing the run connected with the boundary of the circumscribed rectangle rect' and all 4-connected domain runs formed by the run in the background region backGroundRegion, the remaining region is the hole region holeRegion.

[0120] The union set of the foreground region region and the hole region holeRegion is obtained to complete the hole filling of region.

[0121] To achieve the above object, the present application further provides an image processing device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the hole filling method based on run-length encoding. The memory, the processor and the computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the hole filling method based on run-length encoding. A main controller can be used to implement, such as a DSP (Digital Signal Process), an FPGA (Field Programmable Gate Array), an MCU (Microcontroller Unit), an SOC (System On Chip) and the like.

[0122] It is worth noting that since the image processing device of the present application is applied to the hole filling method based on run-length encoding, the embodiments of the image processing device of the present application include all the technical solutions of all the embodiments of the hole filling method based on run-length encoding, and the technical effects achieved are also completely the same, which will not be repeated here.

[0123] The above merely describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or the like, which is made based on the technical concept of the present application and the content of the specification and drawings, is included in the patent protection scope of the present application.

Claims

1. A hole filling method based on run-length encoding, characterized by, The method comprises the following steps: obtaining a binary image, performing run-length encoding on the binary image to obtain a foreground region; calculating a circumscribed rectangle of the foreground region, and expanding the boundary of the circumscribed rectangle to the periphery to obtain an expanded circumscribed rectangle and a rectangular region; traversing each row of the rectangular region, and performing a difference set calculation on the current row of the rectangular region and the same row of the foreground region to obtain a row difference set result of the current row; based on the row difference set result of each row, calculating a difference set between the rectangular region and the foreground region to obtain a background region; wherein when the foreground region row is on the left / right side of the rectangular region row and has no overlap, the rectangular region row is retained; when the foreground region row partially overlaps the left / right side of the rectangular region row, the non-overlapping part of the rectangular region row is retained; when the foreground region row is completely contained in the rectangular region row, the non-overlapping part of the rectangular region row and the foreground region row is retained, and the part is divided into a front segment and a rear segment by the foreground region row; when the foreground region row completely contains the rectangular region row, the rectangular region row is discarded; in the background region, removing the run-lengths connected to the boundary of the expanded circumscribed rectangle, and removing all run-lengths connected to these run-lengths through connected domain analysis to obtain a hole region; performing a union operation on the foreground region and the hole region to obtain a foreground region after hole filling.

2. The run-length coding based hole filling method of claim 1, wherein, The specific steps of obtaining a binary image, performing run-length encoding on the binary image to obtain a foreground region comprise: obtaining a binary image, performing run-length encoding on the binary image to obtain a run-length set; defining the run-length set as a foreground region.

3. The run-length coding based hole filling method of claim 1, wherein, The specific steps of calculating a circumscribed rectangle of the foreground region, expanding the boundary of the circumscribed rectangle to the periphery to obtain an expanded circumscribed rectangle and a rectangular region comprise: calculating a circumscribed rectangle of the range covered by the foreground region; expanding the boundary of the circumscribed rectangle to the periphery by one pixel to obtain an expanded circumscribed rectangle; generating a rectangular region based on the expanded circumscribed rectangle.

4. The run-length coding based hole filling method of claim 3, wherein, The circumscribed rectangle of the range covered by the foreground region is the minimum circumscribed rectangle of the foreground region.

5. The run-length coding based hole filling method of claim 3, wherein, The rectangular region comprises the foreground region, a first region not included in the foreground region but covered by the circumscribed rectangle, and a second region expanded to the periphery of the circumscribed rectangle.

6. The run-length coding based hole filling method of claim 1, wherein, The specific steps of removing, in the background region, the run-lengths connected to the boundary of the expanded circumscribed rectangle, and removing all run-lengths connected to these run-lengths through connected domain analysis to obtain a hole region comprise: in the background region, removing the run-lengths connected to the boundary of the expanded circumscribed rectangle; taking the run-lengths connected to the boundary as seed run-lengths, and through 4-connected domain growth, marking all run-lengths connected to the seed run-lengths in the background region as a boundary connected region; removing all run-lengths in the boundary connected region to obtain a hole region.

7. The run-length coding based hole filling method of claim 6, wherein, The run-lengths connected to the boundary of the expanded circumscribed rectangle refer to run-lengths satisfying one of the following conditions: the row number of the run-length is equal to the row where the top edge or the bottom edge of the expanded circumscribed rectangle is located, and the column coordinate is between the left and right boundaries of the expanded circumscribed rectangle; the column number of the run-length is equal to the column where the left edge or the right edge of the expanded circumscribed rectangle is located, and the row coordinate is between the upper and lower boundaries of the expanded circumscribed rectangle.

8. An image processing apparatus characterized by comprising: The method comprises the following steps: A memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program configured to implement the steps of the run-length coding based hole filling method of any one of claims 1 to 7.

Images