Method and device for removing binding holes in scanned image

Abstract

The invention discloses a method and device for removing binding holes in a scanned image, and relates to the technical field of image processing. In the method, the removal device detects a to-be-processed scanning image based on a preset binding hole detection model to obtain a binding hole prediction position; when the background base plate of the scanner is gray, determining whether the minimum value of the three-channel color values of the pixel points of the scanned image is greater than a preset background brightness threshold or whether the maximum value of the three-channel color values is less than a preset threshold; if yes, determining that the pixel points of the scanned image are white points; if not, determining that the pixel points of the scanned image are black points; pixel points of the scanned image are converted into white points or black points, and a binary image is obtained; cutting the predicted positions of the binding holes in the binarized image into sub-images; performing contour searching on the sub-images to obtain a binding hole mask image; and filling and repairing the scanned image based on the binding hole mask image and the background base plate color of the scanner to obtain a target image without binding holes.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for removing binding holes from scanned images. Background Technology

[0002] As a crucial image acquisition device, a scanner converts paper documents into digital image information using photosensitive elements. During the scanning process, if the paper being scanned has binding holes, the scanner's background (usually black, white, or gray) will be captured through these holes, resulting in a hole area in the final digital image that matches the background color. These holes not only have no practical information value but also severely affect the overall aesthetics and subsequent use of the image.

[0003] Related technologies have proposed several heuristic rule-based solutions. A common approach is to first convert the original color image to grayscale and then binarize it to obtain a black-and-white binary image. Subsequently, based on this binary image, pre-defined rules are used to search for and locate binding holes. These rules are typically based on assumptions about the physical characteristics of binding holes; for example, through connected component analysis, regions that conform to a pre-defined size and shape (such as approximately circular) are selected as candidate binding holes. To further improve accuracy, some solutions also incorporate location information, such as assuming that binding holes always appear near the edges of the paper, or that multiple binding holes are roughly distributed along a straight line. Furthermore, some solutions introduce color features, calculating the variance of pixel values ​​and gradient values ​​within candidate regions to aid in the judgment, distinguishing real binding holes from other dark content on the document.

[0004] However, related technologies are generally based on heuristic rules and rely on image binarization to segment binding holes. This method is acceptable for processing regular binding holes against a black background, but when the scanner background is gray or the binding hole shape is complex (such as multiple binding holes overlapping), the pixel values ​​of the hole area are very close to some document content (such as light-colored text or seals). Related technologies cannot effectively separate them based on heuristic rules, which reduces the accuracy of binding hole removal in scanned images. Summary of the Invention

[0005] This application provides a method and apparatus for removing binding holes from scanned images, thereby improving the accuracy of binding hole removal from scanned images.

[0006] In a first aspect, a method and apparatus for removing binding holes from a scanned image are provided. The method, applied to a removal apparatus, includes: the removal apparatus detecting the scanned image to be processed based on a preset binding hole detection model to obtain the predicted position of the binding hole; when the scanner background is gray, the removal apparatus determining whether the minimum value of the three-channel color values ​​of the pixels in the scanned image is greater than a preset background brightness threshold or whether the maximum value of the three-channel color values ​​is less than a preset threshold; if so, the removal apparatus determines that the pixels in the scanned image are white; if not, the removal apparatus determines that the pixels in the scanned image are black; the removal apparatus converts the pixels in the scanned image into white or black points to obtain a binarized image; the removal apparatus cropping the predicted binding hole position in the binarized image into a sub-image; the removal apparatus performing contour lookup on the sub-image to obtain a binding hole mask image; and the removal apparatus filling and repairing the scanned image based on the binding hole mask image and the scanner background color to obtain a target image with the binding hole removed.

[0007] By adopting the above technical solution, the preset binding hole detection model can autonomously learn the deep visual features of binding holes from a large amount of data, rather than relying on preset simple geometric shapes (such as circles) or color thresholds. This allows the preset binding hole detection model to generalize and identify various irregular, damaged, or special-shaped (such as square or elliptical) binding holes, thus exhibiting stronger environmental adaptability. Based on the predicted binding hole location determined by the preset binding hole detection model, for a gray background, by judging whether the minimum and maximum values ​​of the pixel's three-channel color values ​​fall outside a specific range, gray hole areas with brightness between paper and dark text are identified as foreground (black dots), while bright paper backgrounds and extremely dark text strokes are classified as background (white dots). This allows for the generation of a binarized image that highlights the binding holes even in low-contrast scenes, thereby improving the accuracy of binding hole removal in scanned images.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the removal device detecting the scanned image to be processed based on a preset binding hole detection model to obtain the predicted position of the binding hole specifically includes: the removal device performing a standardization preprocessing operation on the scanned image to generate standardized image data; the removal device inputting the standardized image data into the preset binding hole detection model to obtain an initial set of binding hole candidate regions and a confidence score corresponding to each initial binding hole candidate region; the removal device filtering binding hole candidate regions in the initial set of binding hole candidate regions whose confidence scores are less than a preset confidence threshold; and the removal device eliminating redundant binding hole candidate regions with an overlap greater than a preset overlap threshold based on a non-maximum suppression algorithm in the filtered initial set of binding hole candidate regions to obtain the predicted position of the binding hole.

[0009] By employing the above technical solution, the error caused by false detections and low-quality detection results is reduced by filtering out candidate binding hole regions with confidence scores lower than a preset confidence threshold, thereby improving the accuracy of binding hole prediction. A non-maximum suppression algorithm is used to process the filtered initial set of candidate binding hole regions, eliminating redundant candidate regions with overlap exceeding a preset overlap threshold, thus reducing redundant calculations and resource waste in subsequent processing.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the removal device cropping the predicted binding hole position in the binarized image into a sub-image specifically includes: the removal device performing a morphological opening operation of a preset size on the binarized image; the removal device performing connected component analysis on the binarized image after the morphological opening operation to obtain all connected regions; the removal device calculating the area compactness parameter of each connected region; the removal device identifying connected regions with an area greater than a preset area threshold and a compactness within a preset range as potential binding hole regions; the removal device matching and verifying the position information of the potential binding hole regions with the predicted binding hole position to determine the final valid binding hole position; and the removal device cropping the binarized image based on the valid binding hole position to obtain a sub-image.

[0011] By employing the above technical solution, a morphological opening operation of a preset size is performed on the binarized image, breaking the subtle connections between the binding hole and surrounding text strokes or graphic elements. This avoids misidentifying the binding hole and document content as a single entity in subsequent connected component analysis, thereby improving the accuracy of binding hole region separation. Connected component analysis is performed on the processed binarized image to obtain all connected regions. Then, by calculating the area compactness parameter of each connected region, a geometric feature screening mechanism for the binding hole region is established. The condition that the area is greater than a preset area threshold reduces noise points and subtle interference, improving the accuracy of obtaining candidate binding hole regions. Matching and verifying the location information of potential binding hole regions with the predicted binding hole locations obtained from deep learning detection further improves the reliability of binding hole localization. Based on the effective binding hole locations, precise region cropping is performed on the binarized image to obtain sub-images, avoiding contour searching of the entire image and thus improving processing efficiency.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the removal device performing a morphological opening operation of a preset size on the binarized image specifically includes: the removal device calculating the optimal size of the morphological structural element based on the average size of the binding hole region in the predicted binding hole position; the removal device constructing an elliptical structural element of the optimal size; and the removal device using the elliptical structural element to sequentially perform morphological erosion and morphological dilation operations on the binarized image to break the fine connection between the binding hole and the text strokes.

[0013] By adopting the above technical solution, compared with traditional rectangular or circular structural elements, elliptical structural elements better match the typical geometric shape features of binding holes. When performing morphological operations, they can more accurately maintain the original contour features of the binding holes and reduce shape distortion. Erosion operations can break the tiny connecting bridges between the binding holes and text strokes, reducing connection noise that may affect the accuracy of connected component analysis. Subsequent dilation operations can restore the original size and shape features of the main binding hole area, reducing the detection omission rate caused by excessive shrinkage of the binding hole area.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the removal device performing contour search on the sub-image to obtain a binding hole mask image specifically includes: the removal device extracting the set of coordinate points of all closed contours in the sub-image based on an edge contour tracking algorithm; the removal device taking closed contours with an enclosing area greater than a preset area threshold as candidate binding hole contours; the removal device mapping the coordinate points of the candidate binding hole contours from the coordinate system of the sub-image back to the coordinate system of the scanned image; the removal device initializing a zero-mask image with the same size as the scanned image; and the removal device marking the coordinate points of the candidate binding hole contours in the zero-mask image as 255 to obtain the binding hole mask image.

[0015] By adopting the above technical solution, the removal device extracts the set of coordinate points of all closed contours in the sub-image based on the edge contour tracking algorithm, providing complete boundary information and topological connection relationship. It uses closed contours with an enclosing area greater than a preset area threshold as a screening mechanism for candidate binding hole contours, reducing interference caused by noise points, small spots and other non-binding hole areas in the image, and improving the accuracy and reliability of binding hole contour recognition.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the step of the removal device filling and repairing the scanned image based on the binding hole mask image and the background color of the scanner to obtain a target image with removed binding holes specifically includes: the removal device determining whether to use a specified color fill mode or a neighboring background color fill mode according to the fill mode parameters input by the scanner device; when using the specified color fill mode, the removal device replacing the pixel value at the position corresponding to the pixel with a pixel value of 255 in the scanned image and the binding hole mask image with a preset color value; when using the neighboring background color fill mode, the removal device calculating the three-channel color average of non-binding hole pixels within a preset width annular area around each binding hole; the removal device using the three-channel color average as the fill color of the corresponding binding hole area; the removal device replacing the pixel value of the binding hole position in the scanned image based on the fill color; and the removal device performing edge smoothing processing on the filled and repaired target image to eliminate abrupt changes between the filled area and the surrounding pixels, generating a target image with removed binding holes.

[0017] By employing the above technical solution, in the specified color fill mode, the corresponding positions of pixels with a pixel value of 255 in the scanned image and the binding hole mask image are replaced with preset color values. This is suitable for scanned images with relatively uniform background colors and can quickly complete the binding hole removal process. In the nearest neighbor background color fill mode, the average three-channel color of non-binding hole pixels within a preset width annular area around each binding hole is calculated as the fill color. This adaptive color calculation method can determine the most matching fill color value based on the actual background color characteristics around the binding hole, reducing the color difference problem that may be caused by fixed color fill and making the repaired image more natural and harmonious visually.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after the removal device performs edge smoothing processing on the filled and repaired target image to eliminate abrupt changes between the filled area and surrounding pixels, generating a target image with the binding holes removed, the method further includes: the removal device extracting boundary pixels of each binding hole area in the target image; the removal device calculating a color difference metric between each boundary pixel and its adjacent non-binding hole pixels; the removal device using boundary pixels with color difference metric values ​​greater than a preset difference threshold as boundary pixels to be optimized; the removal device performing multi-scale Gaussian filtering processing on the boundary pixels to be optimized to generate candidate fill color values ​​with different degrees of blur; the removal device establishing a rectangular neighborhood window with a preset pixel radius centered on the boundary pixels to be optimized; the removal device extracting effective pixels within the rectangular neighborhood window, excluding the binding hole areas, as a texture reference pixel set; and the removal device selecting the color value from the candidate fill color values ​​that best matches the texture features of the texture reference pixel set for secondary filling.

[0019] By employing the above technical solution, the removal device identifies boundary pixels with color difference values ​​exceeding a preset difference threshold as the boundary pixels to be optimized, eliminating the need for repeated processing of the entire repair area and improving the efficiency and targeting of the optimization process. Multi-scale Gaussian filtering is performed on the boundary pixels to be optimized to generate candidate fill color values ​​with varying degrees of blur, providing diverse color selection schemes. Different scales of filtering achieve a range of color fusion effects from fine to coarse, offering a rich selection of candidates for optimal color matching. The color value that best matches the texture features of the texture reference pixel set is selected from the candidate fill color values ​​for secondary filling, ensuring that the final repair effect maintains a high degree of consistency with the surrounding area in both color and texture dimensions. This multi-layered, refined optimization mechanism improves the visual quality of the binding hole removal and enhances the quality of the final target image.

[0020] In a second aspect, embodiments of this application provide a removal apparatus, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the removal apparatus to perform the method as described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a removal device, cause the removal device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a removal apparatus, cause the removal apparatus to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the removal apparatus provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Because the preset binding hole detection model can autonomously learn the deep visual features of binding holes from a large amount of data, rather than relying on preset simple geometric shapes (such as circles) or color thresholds, it can generalize and identify various irregular, damaged, or special-shaped binding holes (such as squares and ovals), thus exhibiting stronger environmental adaptability. Based on the predicted binding hole location determined by the preset binding hole detection model, for gray backgrounds, by judging whether the minimum and maximum values ​​of the pixel's three-channel color values ​​fall outside a specific range, gray hole areas with brightness between paper and dark text are identified as foreground (black dots), while bright paper backgrounds and extremely dark text strokes are classified as background (white dots). This allows for the generation of a binarized image that highlights the binding holes even in low-contrast scenes, thereby improving the accuracy of binding hole removal in scanned images.

[0025] 2. The removal device extracts the coordinate point set of all closed contours in the sub-image based on the edge contour tracking algorithm, providing complete boundary information and topological connection relationship. It uses closed contours with an enclosing area greater than a preset area threshold as a screening mechanism for candidate binding hole contours, which reduces the interference caused by noise points, small spots and other non-binding hole areas in the image, and improves the accuracy and reliability of binding hole contour recognition.

[0026] 3. The removal device identifies boundary pixels with color difference values ​​exceeding a preset difference threshold as the boundary pixels to be optimized, eliminating the need for repeated processing of the entire repair area and improving the efficiency and targeting of the optimization process. Multi-scale Gaussian filtering is applied to the boundary pixels to be optimized to generate candidate fill color values ​​with varying degrees of blur, providing diverse color selection schemes. Different scales of filtering achieve a range of color fusion effects from fine to coarse, offering a rich selection of candidates for optimal color matching. The color value that best matches the texture features of the texture reference pixel set is selected from the candidate fill color values ​​for secondary filling, ensuring that the final repair effect maintains a high degree of consistency with the surrounding area in both color and texture dimensions. This multi-layered, refined optimization mechanism improves the visual quality of the binding hole removal and enhances the quality of the final target image. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a method for removing binding holes from a scanned image according to an embodiment of this application.

[0028] Figure 2 This is a schematic diagram of the physical device structure of the removal device in the embodiments of this application. Detailed Implementation

[0029] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0030] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0031] This application provides a method and apparatus for removing binding holes from scanned images, thereby improving the accuracy of binding hole removal from scanned images.

[0032] Please see Figure 1 This is a flowchart illustrating a method for removing binding holes from a scanned image according to an embodiment of this application.

[0033] S101, The removal device detects the scanned image to be processed based on a preset binding hole detection model to obtain the predicted position of the binding hole.

[0034] The removal device refers to a computing device specifically designed for processing the task of removing binding holes from scanned images. The preset binding hole detection model represents a deep learning model trained on a large amount of data, capable of recognizing binding holes. For example, a convolutional neural network (CNN)-based object detection model (such as YOLO) can learn and generalize the visual features of binding holes from complex image backgrounds, not limited to specific geometric shapes. The scanned image to be processed refers to a digitized document image containing binding holes acquired by a scanner. The predicted binding hole location represents the coordinate information of the binding hole in the image, as output by the detection model, and is represented by a rectangular bounding box.

[0035] Specifically, first, the removal device performs standardization preprocessing on the input scanned image. This includes scaling the image size to the standard size used during model training (e.g., 640×640 pixels) and normalizing the pixel values ​​(e.g., converting from the integer range [0, 255] to the floating-point range [0, 1]) to improve the stability of the model input. Next, the standardized image data is input into a preset binding hole detection model for inference computation. The preset binding hole detection model outputs initial candidate boxes and a confidence score for each box, representing the probability that the model considers the box to contain a binding hole. Subsequently, to filter out most invalid, low-quality candidate boxes, the device discards all candidate boxes with confidence scores below a preset threshold (e.g., 0.5). Finally, since a single binding hole may be detected simultaneously by multiple highly overlapping candidate boxes, the removal device performs a non-maximum suppression (NMS) algorithm: in overlapping boxes, the one with the highest confidence is retained, and other redundant boxes with an overlap greater than a preset threshold (e.g., 0.45) are removed, ultimately obtaining a set of predicted binding hole locations.

[0036] In some embodiments, the construction and deployment of the binding hole detection model can be achieved in several ways: Optionally, firstly, scanned image samples containing binding holes of various shapes and sizes are collected, and bounding boxes of the binding holes are annotated using annotation tools such as labelImg. Then, the annotated data is divided into training and testing sets in an 8:2 ratio, and a YOLO model is trained based on the PyTorch deep learning framework. After training, the PyTorch format model is converted to ONNX format for deployment. Optionally, a pre-trained general object detection model can also be used as a foundation, and fine-tuned on the binding hole dataset through transfer learning. Firstly, the backbone network parameters of the model are frozen, and only the detection head part is trained. Then, more layers are gradually unfrozen for end-to-end training. Finally, model performance is optimized through model quantization and pruning techniques to meet real-time processing requirements.

[0037] S102. When the background plate of the scanner is gray, the removal device determines whether the minimum value of the three-channel color value of the pixel of the scanned image is greater than the preset background brightness threshold or whether the maximum value of the three-channel color value is less than the preset threshold.

[0038] The scanner's background plate refers to the internal structure of the scanner, located opposite the photosensitive element, used to provide a background for the scanned document. A gray background plate indicates that the plate color is gray. Compared to a black or white background, the binding holes on a gray background are closer to the document content in pixel values, increasing the difficulty of separation. The three-channel color value refers to the numerical values ​​of the blue, green, and red color components of each pixel in a color image. The preset background brightness threshold is pre-set according to the characteristics of the scanner's background plate and can be set to the maximum brightness value of the background color. For example, a normal background color has a brightness between 130-140; in an exemplary embodiment, the preset background brightness threshold can be set to 140. The preset threshold represents a fixed value used to determine whether a pixel is dark content; in an exemplary embodiment, it can be set to 40.

[0039] Specifically, this step is performed after the predicted location of the binding hole is obtained. Its fundamental purpose is not the traditional separation of content from background, but rather to identify the binding hole as the foreground (black dot) in a complex environment (containing white paper, dark text, gray holes, possible light-colored content, and possible stains), while classifying all other elements (paper and text, etc.) as background (white dots). If the darkest channel value of a pixel is greater than a preset background brightness threshold, then it must be a high-brightness area, such as white paper or a very light stain, and should be identified as a white dot. If the brightest channel value of a pixel is darker than a preset threshold, then it must be an extremely dark area, such as black text strokes, and should be identified as a white dot. Only when a pixel is neither bright enough nor dark enough—that is, when the minimum value of the three-channel color values ​​of a pixel in the scanned image is not greater than the preset background brightness threshold, and the maximum value of the three-channel color values ​​is not less than the preset threshold of 40—is the pixel identified as a black dot. This characteristic of brightness falling precisely in the middle is a typical feature of the binding hole area (including its shadow) on a gray background.

[0040] In some embodiments, adaptive pixel classification against a gray background can be achieved in several ways: Optionally, firstly, color statistical analysis is performed on the entire scanned image to calculate the distribution histogram of gray values ​​in the image. A peak detection algorithm is then used to identify the main distribution range of background gray values. Next, a preset background brightness threshold is dynamically adjusted based on the statistical results. Finally, the adjusted threshold is applied to each pixel for classification. Optionally, a local adaptive approach can also be used. The image is divided into multiple sub-regions, and local background brightness features are calculated for each sub-region. A corresponding judgment threshold is set for each sub-region based on the local features. Then, black-and-white classification of pixels is performed independently within each sub-region. Finally, the results from all sub-regions are merged to form a complete classification result.

[0041] S103, The removal device determines that a pixel in the scanned image is a white pixel.

[0042] Specifically, this step is executed when the condition in step S102 is met. When the removal device detects that the minimum value of the three-channel color value of a pixel in the scanned image is greater than a preset background brightness threshold, or the maximum value of the three-channel color value is less than a preset threshold, the removal device identifies and marks the pixel as a white point.

[0043] S104, The removal device determines that the pixels in the scanned image are black dots.

[0044] Specifically, this step is executed when the condition judgment in step S102 is negative. When the removal device detects that the minimum value of the three-channel color value of a certain pixel in the scanned image is not greater than the preset background brightness threshold and the maximum value of the three-channel color value is not less than the preset threshold of 40, the removal device identifies and marks the pixel as a black point.

[0045] S105, The removal device converts the pixels of the scanned image into white or black dots to obtain a binarized image.

[0046] Pixel conversion refers to the process of converting each pixel in the original scanned image into a binary representation based on the aforementioned classification results. A binary image is a single-channel grayscale image containing only two pixel values ​​(0 and 255), used to simplify subsequent contour detection and morphological processing.

[0047] Specifically, this step is performed after all pixels have been classified as black or white. When the background is gray, the removal device iterates through each pixel in the scanned image, and based on the classification results of steps S103 and S104, sets all pixels marked as black to 0 (black foreground) in the binarized image, and sets all pixels marked as white to 255 (white background). This conversion process creates a binarized image that is exactly the same size as the original scanned image but contains only black and white. In this binarized image, the binding hole is presented as a black area, while the background area is presented as white, providing a clear foreground-background separation effect for subsequent morphological processing and contour finding.

[0048] In some embodiments, when the scanner background is black, the removal device can invoke a standard OTSU binarization function to process the entire grayscale scanned image to obtain a globally binarized image. In such high-contrast scenarios, OTSU can effectively separate white paper from the black background (and holes).

[0049] S106, The removal device crops the predicted binding hole position in the binarized image into a sub-image.

[0050] The predicted binding hole location refers to the coordinate information of the bounding box of the binding hole rectangle output by the YOLO detection model in step S101. A sub-image represents a local image region containing one or more binding holes, cropped from the binarized image.

[0051] Specifically, this step is performed after obtaining the binarized image. The removal device first calculates the optimal size of an elliptical structural element based on the average size of the binding hole region detected in step S101. Then, it constructs this elliptical structural element and performs a morphological opening operation on the binarized image. This operation, involving erosion followed by dilation, breaks the fine connections between the binding hole and surrounding text strokes while maintaining the integrity of the main binding hole region. Next, the removal device performs connected component analysis on the processed binarized image, identifying all interconnected foreground pixel regions and calculating the area compactness parameter (also called roundness or shape factor) for each connected region. For a standard circle, the area compactness parameter is 1; the more irregular the shape, the smaller the area compactness parameter. By setting a preset compactness range (e.g., [0.6, 1.0]), regions that more closely resemble binding holes in shape can be retained, while slender, irregular residual text strokes or interference are excluded. The removal device filters out potential binding hole regions from all connected regions based on screening criteria that the area is greater than a preset threshold (e.g., 1000 pixels) and the compactness is within a preset compactness range. Subsequently, the removal device performs spatial matching verification between the location information of these potential binding hole regions and the YOLO detection locations obtained in step S101. Only those connected regions that are both inside or near the YOLO prediction box and meet the morphological and geometric screening criteria are confirmed as the final valid binding hole locations, and the binarized image is cropped accordingly to obtain sub-images.

[0052] In some embodiments, the cutting of the binding hole area can be achieved in several ways: Optionally, a multi-scale morphological processing method can also be used. Specifically, after obtaining the predicted binding hole positions (i.e., a series of rectangular bounding boxes) in S101, the removal device extracts the width and height of each predicted binding hole rectangle. Subsequently, the average width and average height are obtained by taking the arithmetic mean of all width and height values. The size selection of the morphological structural elements is crucial: if the size is too small, it may not be able to effectively break the connection with the text. If the size is too large, it will over-erode the binding hole itself, resulting in contour distortion or loss. Therefore, an optimal size should be positively correlated with but slightly smaller than the average size of the binding hole. For example, in one specific embodiment, the optimal size can be set as a preset proportion of the average size, such as setting the optimal width to 0.7 times the average width and the optimal height to 0.7 times the average height. Next, based on the optimal width and height, the removal device calls an image processing library (such as the `getStructuringElement` function in OpenCV) and specifies the shape parameter as `MORPH_ELLIPSE` to generate an elliptical structuring element. The ellipse is chosen because it better approximates the geometry of most binding holes. Finally, the removal device uses this elliptical structuring element to perform morphological erosion and morphological dilation operations on the binarized image. The erosion operation shrinks the foreground region (black area), effectively eroding away small text strokes or noise points connected to the edge of the binding hole. The subsequent dilation operation restores the size of the eroded binding hole body, but the previously broken small connections are not reconnected because they have already been separated. Through this series of operations, the small connections between the binding hole body and the text strokes are broken while preserving the integrity of the binding hole's outline. As an alternative implementation, a multi-scale morphological processing approach can also be used. The removal device can pre-construct multiple elliptical structuring elements of different sizes (e.g., sizes increasing from small to large) and use them to perform opening operations on the binarized image. Subsequently, connected component analysis and feature calculation are performed independently on the processing results at each scale. An optimal processing scale is automatically selected by comparing the stability of the target connected components at different scales (e.g., at which scale features such as area and compactness change the least). Based on the processing results at this optimal scale, subsequent binding hole region cropping is performed. This method is better suited to situations where there are multiple binding holes of significantly different sizes in the image.

[0053] S107. The removal device performs contour search on the sub-image to obtain the binding hole mask image.

[0054] Contour detection refers to a computer vision algorithm that detects and extracts the boundaries of a target object in a binary image. The binding hole mask image represents a binary image of the same size as the original scanned image, where the binding hole region has a pixel value of 255 and the background region has a pixel value of 0.

[0055] Specifically, this step is performed after obtaining the sub-image. The removal device uses an edge contour tracking algorithm to perform contour detection on the sub-image. This algorithm scans the foreground pixels in the binary image, identifies and tracks the connection relationships between adjacent pixels, and extracts the complete set of coordinate points for all closed contours. To eliminate noise and small interference areas, the removal device calculates the enclosing area of ​​each closed contour and filters contours with areas greater than a preset threshold as candidate binding hole contours. This threshold is usually set according to the typical size of a binding hole. Since the sub-image is a local region cropped from the original scanned image, the removal device needs to map the coordinate points of the candidate contours from the local coordinate system of the sub-image back to the global coordinate system of the scanned image. This is achieved by adding the offset of the sub-image in the scanned image to the original coordinates. Finally, the removal device creates a mask image with all pixel values ​​of 0 that is exactly the same size as the scanned image, and marks the mapped candidate binding hole contour coordinate points as 255 in the mask image, forming a binding hole mask image.

[0056] In some embodiments, contour extraction and mask generation can be achieved in multiple ways: Optionally, the removal device employs a multi-level contour detection strategy. First, the outermost contour is extracted using the RETR_EXTERNAL mode to avoid interference from internal holes. Then, the RETR_LIST mode is used to extract contour information from all levels for each outer contour. The true binding hole boundary is identified through contour hierarchy relationship analysis. Next, the Douglas-Peucker algorithm is used to approximate the contour with polygons, reducing the number of coordinate points. Finally, a contour filling algorithm is used to completely fill the simplified contour into the mask image. Optionally, a contour quality assessment method can also be used. The removal device considers not only the contour area but also comprehensively evaluates the geometric features of the contour, such as roundness, convexity, and regularity. The minimum circumcircle and minimum circumrectangle of the contour are calculated. The contour quality is judged by comparing the ratio of the contour area to the area of ​​the circumscribed shape. Only contours with a quality score exceeding a preset threshold are considered valid binding hole contours. Then, anti-aliasing filling technology is used to draw the contour region in the mask image.

[0057] S108, the removal device fills and repairs the scanned image based on the binding hole mask image and the background plate color of the scanner to obtain the target image with the binding hole removed.

[0058] Filling repair refers to the image restoration process of replacing pixel values ​​in the binding hole area with specific colors or texture information. The background plate color represents the color characteristics of the physical background plate of the scanner device and is used to guide the color filling of the binding hole area.

[0059] Specifically, this step is performed after obtaining the binding hole mask image. The removal device first determines the specific repair strategy based on the fill mode parameters input from the scanner. In the specified color fill mode, the removal device directly replaces the pixel position marked as 255 in the binding hole mask image of the scanned image with a preset color value, which is usually set to an RGB value that matches the background color of the scanner. In the nearest neighbor background color fill mode, the removal device adopts a more intelligent adaptive fill strategy: for each detected binding hole area, it calculates the average RGB three-channel color of all non-binding hole pixels within a ring area of ​​a preset width (e.g., 20-30 pixels) around it, uses this average value as the dedicated fill color for the corresponding binding hole area, and then replaces the pixel value of the binding hole position based on the calculated fill color. After completing the basic fill, the removal device performs edge smoothing processing on the repair area, using Gaussian filtering or bilateral filtering techniques to eliminate color abrupt changes between the filled area and surrounding pixels, improving the naturalness of the repair effect. To further improve the repair quality, the removal device will also perform secondary optimization processing: extract the boundary pixels of each binding hole area, calculate the color difference metric between the boundary pixels and the adjacent non-binding hole pixels, perform multi-scale Gaussian filtering on the boundary pixels with difference values ​​exceeding a preset threshold to generate candidate fill color values, then establish a rectangular neighborhood window to extract the texture reference pixel set, and finally select the color value that best matches the texture features for refined secondary filling.

[0060] In some embodiments, the quality of binding hole filling repair can be improved in several ways: Optionally, the removal device can employ a layered repair strategy. First, morphological closing operations are used to expand the binding hole mask region to ensure complete coverage of the binding hole edges. Then, the filling process is divided into two stages: kernel filling and edge transition. A uniform fill color is used in the kernel region, and a gradient transition technique is used in the edge region. Finally, anisotropic diffusion filtering is applied to optimize the texture consistency of the entire repair area. Optionally, a sample-based texture synthesis method can also be used. The removal device establishes multiple sampling windows around the binding hole to extract background texture samples, uses a texture synthesis algorithm to generate filling content consistent with the texture pattern of the surrounding area, and seamlessly integrates the synthesized texture into the original image using Poisson image editing technology to achieve a more visually natural repair effect.

[0061] In the above embodiments, the preset binding hole detection model quickly locates the approximate position of the binding hole in the scanned image, improving the processing efficiency of extracting the binding hole position. For the specific application scenario of a scanner with a gray background, a specialized binarization judgment logic is established by analyzing the relationship between the minimum and maximum values ​​of the three-channel color values ​​of pixels and a preset threshold. This binarization judgment logic has stronger environmental adaptability and higher segmentation accuracy compared to the traditional fixed threshold binarization method. By cropping the predicted binding hole position in the binarized image into sub-images, interference from irrelevant areas is reduced, improving the accuracy of contour finding. A filling and repair strategy based on the binding hole mask image and the scanner background color makes the final target image visually close to the original scan effect without binding holes.

[0062] The above describes a method for removing binding holes in a scanned image according to an embodiment of this application. The following describes an exemplary removal device 200 provided in an embodiment of this application.

[0063] Figure 2 This is a schematic diagram of an exemplary hardware structure of the removal device 200 provided in an embodiment of this application. In some embodiments, the removal device 200 is a computer device. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements a method for removing binding holes in a scanned image according to an embodiment of this application.

[0064] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0065] In some embodiments of this application, a computer-readable storage medium is also provided, including instructions that, when executed on the removal device 200, cause the removal device 200 to perform a method for removing binding holes in a scanned image according to an embodiment of this application.

[0066] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0067] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0068] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0069] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for removing binding holes from a scanned image, characterized in that, Applied to a removal device, the method includes: The removal device detects the scanned image to be processed based on a preset binding hole detection model to obtain the predicted position of the binding hole. When the background of the scanner is gray, the removal device determines whether the minimum value of the three-channel color value of the pixel in the scanned image is greater than a preset background brightness threshold or whether the maximum value of the three-channel color value is less than a preset threshold. If so, the removal device determines that the pixel of the scanned image is a white point; If not, the removal device determines that the pixels in the scanned image are black dots; The removal device converts the pixels of the scanned image into black or white dots to obtain a binarized image. The removal device crops the predicted position of the binding hole in the binarized image into a sub-image; The removal device performs contour search on the sub-image to obtain a binding hole mask image; The removal device fills and repairs the scanned image based on the binding hole mask image and the background color of the scanner to obtain a target image with the binding holes removed.

2. The method according to claim 1, characterized in that, The step of the removal device detecting the scanned image to be processed based on a preset binding hole detection model to obtain the predicted position of the binding hole specifically includes: The removal device performs a standardization preprocessing operation on the scanned image to generate standardized image data; The removal device inputs the standardized image data into the preset binding hole detection model to obtain an initial set of binding hole candidate regions and a confidence score corresponding to each initial binding hole candidate region. The removal device filters out binding hole candidate regions in the initial binding hole candidate region set whose confidence scores are less than a preset confidence threshold. The removal device eliminates redundant binding hole candidate regions with an overlap greater than a preset overlap threshold by using a non-maximum suppression algorithm on the filtered initial binding hole candidate region set, thereby obtaining the predicted binding hole position.

3. The method according to claim 1, characterized in that, The step of the removal device cropping the predicted binding hole position in the binarized image into a sub-image specifically includes: The removal device performs a morphological opening operation of a preset size on the binarized image; The removal device performs connected component analysis on the binarized image after morphological opening operation to obtain all connected regions. The removal device calculates the area compactness parameters of each connected region; The removal device uses connected regions with an area greater than a preset area threshold and a compactness within a preset range as potential binding hole regions. The removal device matches and verifies the location information of the potential binding hole area with the predicted binding hole location to determine the final valid binding hole location. The removal device performs region cropping on the binarized image based on the effective binding hole position to obtain the sub-image.

4. The method according to claim 3, characterized in that, The step of the removal device performing a morphological opening operation of a preset size on the binarized image specifically includes: The removal device calculates the optimal size of the morphological structural element based on the average size of the binding hole region in the predicted binding hole position; The removal device constructs the elliptical structural element of the optimal size; The removal device uses the elliptical structuring element to sequentially perform morphological erosion and morphological dilation operations on the binarized image, breaking the fine connection between the binding hole and the text strokes.

5. The method according to claim 1, characterized in that, The step of the removal device performing contour searching on the sub-image to obtain a binding hole mask image specifically includes: The removal device extracts the set of coordinate points of all closed contours in the sub-image based on an edge contour tracking algorithm; The removal device uses closed contours with an area larger than a preset area threshold as candidate binding hole contours. The removal device maps the coordinates of the candidate binding hole outline from the coordinate system of the sub-image back to the coordinate system of the scanned image; The removal device initializes a zero-mask image of the same size as the scanned image; The removal device marks the coordinates of the candidate binding hole outline in the all-zero mask image as 255, thus obtaining the binding hole mask image.

6. The method according to claim 1, characterized in that, The step of the removal device filling and repairing the scanned image based on the binding hole mask image and the background color of the scanner to obtain the target image with the binding holes removed specifically includes: The removal device determines whether to use a specified color fill mode or a neighboring background color fill mode based on the fill mode parameters transmitted from the scanner device. When a specified color fill mode is used, the removal device replaces the pixel value at the position corresponding to the pixel with a pixel value of 255 in the scanned image and the binding hole mask image with a preset color value. When using the nearest neighbor background color fill mode, the removal device calculates the three-channel color average of non-binding hole pixels within a preset width annular area around each binding hole; The removal device uses the average value of the three-channel colors as the fill color for the corresponding binding hole area; The removal device replaces pixel values ​​at the binding hole positions in the scanned image based on the fill color; The removal device performs edge smoothing processing on the target image after filling and repair, eliminating abrupt changes between the filled area and surrounding pixels, and generating a target image with the binding hole removed.

7. The method according to claim 6, characterized in that, After the removal device performs edge smoothing processing on the filled and repaired target image to eliminate abrupt changes between the filled area and surrounding pixels, generating a target image with the binding hole removed, the method further includes: The removal device extracts the boundary pixels of each binding hole region in the target image; The removal device calculates a color difference metric between each boundary pixel and its adjacent non-binding hole pixels. The removal device identifies boundary pixels whose color difference metric value is greater than a preset difference threshold as boundary pixels to be optimized. The removal device performs multi-scale Gaussian filtering on the boundary pixels to be optimized to generate candidate fill color values ​​with different degrees of blur. The removal device establishes a rectangular neighborhood window with a preset pixel radius centered on the boundary pixel to be optimized; The removal device extracts the effective pixels within the rectangular neighborhood window, excluding the binding hole area, as a texture reference pixel set. The removal device selects the color value that best matches the texture features of the texture reference pixel set from the candidate fill color values ​​for secondary filling.

8. A removal device, characterized in that, The removal apparatus includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the removal apparatus to perform the method as described in any one of claims 1-7.

9. A computer program product containing instructions, characterized in that, When the computer program product is run on the removal device, the removal device performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the removal device, the removal device performs the method as described in any one of claims 1-7.

Images