A handwritten font removal method based on convolution adaptive denoising

Abstract

The present invention relates to a method for removing handwritten fonts based on convolution adaptive denoising, and the steps are as follows: S1 handwriting and printing pixel classification: marking a number of image samples, classifying the handwritten part and the printed part in the marked image samples by pixels, and then removing the handwritten part from the image samples; S2 handwriting mask image processing: obtaining a handwriting mask image according to the classification results of the handwritten part and the printed part, and then binarizing the mask image to obtain a processed filled mask image; S3 outer contour convolution denoising: performing a sampling convolution operation on the filled mask image to denoise the outer contour of the handwritten part in the image sample; S4 median filtering denoising: applying median filtering to the handwriting mask image, and selecting the median as the critical value of filtering; S5 adaptive threshold analysis: performing adaptive threshold analysis on the image sample after filtering to obtain a grayscale adaptive threshold; S6 binarization denoising to obtain a handwriting removed image.

Description

Technical Field

[0001] The invention belongs to the technical field of computer image processing, and in particular relates to a handprint separation method based on adaptive denoising, which is used to solve the problem of removing handwritten fonts. Background Art

[0002] With the promotion of intelligent technology, various text recognition products have emerged, and the accompanying image processing problems have also emerged in an endless stream. In the field of image processing technology, there are many processing methods for image denoising, but combined with actual product needs, there are still many problems. The current image denoising methods include: local adaptive denoising method and global adaptive denoising method.

[0003] In the field of text image recognition, popular processing methods are not fully applicable to the denoising of actual handwritten font images. Considering that different colored pens have different pixel grayscale values ​​when writing, the traditional denoising methods based on grayscale values ​​cannot effectively remove handwritten fonts in such images. In order to completely remove the handwritten font pixels and denoise some of the remaining handwritten pixels, a combination of traditional and deep methods is particularly important. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a handwriting font removal method based on convolution adaptive denoising for removing handwriting fonts in images, which is mainly used to solve image problems that traditional methods cannot handle and achieve the purpose of completely removing handwriting fonts as much as possible.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: the handwritten font removal method based on convolution adaptive denoising specifically comprises the following steps:

[0006] S1 Handwritten and printed pixel classification: label several image samples, classify the handwritten and printed parts in the labeled image samples by pixels, and then remove the handwritten parts from the image samples;

[0007] S2 Handwriting mask image processing: According to the classification results of the handwriting part and the printed part, a handwriting mask image is obtained, and then the mask image is binarized to obtain a processed filling mask image;

[0008] S3: Outer contour convolution denoising: by performing a sampling convolution operation on the filled mask image obtained in step S2, the outer contour of the handwritten part in the image sample is denoised;

[0009] S4 median filter denoising: Apply median filter to the handwritten mask image, set the grid size of the median filter, sort the grayscale values ​​of the pixels covered in the grid in descending order, and select the median as the critical value of the filter;

[0010] S5 adaptive threshold analysis: in order to remove the residual handwritten pixels in the image sample, the image sample after filtering is adaptively analyzed to obtain a grayscale adaptive threshold;

[0011] S6 Binarization denoising: Binarize the image according to the adaptive threshold value obtained in step S5. After the processing, the background part of the image is completely white, that is, the gray value is 255, and the printed part is completely black, that is, the gray value is 0, so as to obtain the final handwriting removed image.

[0012] Using the above technical solution, firstly, pixel classification technology is used to classify handwritten and printed pixels into two categories; secondly, the classified handwritten pixels are removed from the image; then, convolution denoising is performed on the remaining handwritten pixels; finally, adaptive threshold calculation and binarization processing are performed on the entire image to achieve the purpose of completely removing handwriting.

[0013] As a preferred technical solution of the present invention, in step S1, a semantic segmentation network is used to classify the pixels of the handwritten part and the printed part respectively, and the handwritten part is preliminarily removed from the image sample.

[0014] As a preferred technical solution of the present invention, after the mask image is binarized in step S2, the pixel value of the handwritten part is modified to make the handwritten part white, and then the pixel value of the non-handwritten part is set, and then the hole filling algorithm is used to fill the holes in the handwritten part to obtain a processed filled mask image; wherein the specific steps of using the hole filling algorithm are:

[0015] S21: Assuming that the original mask image is A, the original mask image A is extended outward by one to two pixels, and the pixel values ​​of the original mask image are filled with 0, that is, as the background color and marked as B;

[0016] S22: fill the pixel value of the large background of B to 255 and mark it as C;

[0017] S23: Crop the filled mask image to the original image size, remove the extended area, and mark it as D;

[0018] S24: Then invert D and add it to A to obtain a processed filling mask image.

[0019] As a preferred technical solution of the present invention, the specific steps of step S3 are:

[0020] S31: First, perform a downsampling convolution operation on the mask image obtained in step S2. The convolution kernel size is set to 3, and the convolution matrix is ​​recorded as: The a value in the matrix is ​​non-zero;

[0021] S32: performing a dot product operation on the convolution matrix and the mask image, and the result of the dot product operation between the convolution matrix and the pixel at the zero value position in the mask image is still zero, and the result of the dot product operation between the pixel at the non-zero value position in the mask image is non-zero;

[0022] S33: Fill the pixels at the non-zero value positions to the upsampling layer according to the convolution kernel size to obtain a filled upsampling convolution image;

[0023] S34: Finally, all pixel values ​​at non-zero value positions in the upsampled convolution image are set to 255, and a matrix addition operation is performed with the image sample, thereby achieving the purpose of denoising the outer contour of the handwritten part in the image sample.

[0024] As a preferred technical solution of the present invention, the specific steps of the adaptive threshold analysis in step S5 are:

[0025] S51: First, the image matrix is ​​traversed according to the gray value range 0-255, and the distribution number of gray values ​​is counted, which is recorded as: X = {x0, x1, ..., x 255}; where x0 represents the number of pixels with a gray value of 0, and the others are similar;

[0026] S52: Secondly, the grayscale pixel probability distribution of all pixels is counted, which is denoted as: P = {p0, p1, ..., p 255}; where p0 represents the percentage of pixels with a gray value of 0, and other probability element values ​​are expressed in the same way. The formula is 1-1:

[0027]

[0028] S53: Finally, for the distribution X of the gray value, a polynomial curve is fitted, as shown in formula 1-2. In order to make the curve fitting closer to the discrete points of the real distribution, the value of M is 10, and w is a parameter;

[0029]

[0030] Then find the first-order partial derivative of function f, and find the extreme point based on the monotonicity of the function.

[0031] The maximum point set is denoted as: f max ={f(x a ), f(x b ), …}; where a and b represent the first maximum value of x at point a of the function and the second maximum value of x at point b of the function respectively;

[0032] The minimum point set is denoted as: f min ={f(x j ), f(x k), …}; where j and k represent the first minimum point of x at function point j and the second minimum point of x at function point k respectively;

[0033] According to the law of large numbers, the number of pixels with grayscale values ​​is normally distributed. In order to make the calculated adaptive grayscale value more accurate, all points above the trough of the function curve are selected as statistical participating points. For multiple troughs in the actual function curve, the maximum value of the minimum value point set is taken, which is recorded as: max(f min );

[0034] As the dividing line for grayscale pixel count statistics, calculate all pixel points corresponding to the statistical number above the dividing line, and calculate their grayscale mean as the grayscale adaptive threshold;

[0035] If the minimum point set is empty, the probability distribution statistics of grayscale pixels are used as a reference, and the probability values ​​in the probability distribution P are sorted in descending order, denoted as: P1; the pixel points corresponding to the first half of the probability values ​​in P1 are selected, and their grayscale mean is calculated as the grayscale adaptive threshold.

[0036] As a preferred technical solution of the present invention, the semantic segmentation network in step S1 uses Deeplab V3+ and cooperates with the backbone network resnet101 and / or Xception.

[0037] As a preferred technical solution of the present invention, in step S4, the size of the value filter is set to a 3*3 grid.

[0038] Compared with the prior art, the present invention has the following beneficial effects: first, the present invention uses pixel classification technology to classify handwritten and printed pixels; second, remove the classified handwritten pixels from the image; then, perform convolution denoising on the remaining handwritten pixels; finally, perform adaptive threshold calculation and binarization processing on the entire image to achieve the purpose of completely removing handwriting. The present invention is used to solve image problems that traditional methods cannot handle, and can analyze and judge the printed and handwritten pixels of the image to achieve the purpose of completely removing handwriting from the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The technical solution of the present invention is further described below in conjunction with the accompanying drawings:

[0040] Figure 1 It is a flow chart of the handwriting font removal method based on convolution adaptive denoising of the present invention;

[0041] Figure 2 It is an algorithm effect diagram of the handwriting font removal method based on convolution adaptive denoising of the present invention. DETAILED DESCRIPTION

[0042] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The embodiments are only used to explain the present invention and do not limit the protection scope of the present invention.

[0043] Example: Figure 1 As shown, the handwritten font removal method based on convolution adaptive denoising specifically includes the following steps:

[0044] The handwritten font removal method based on convolution adaptive denoising specifically includes the following steps:

[0045] S1 Handwritten and printed pixel classification: label several image samples, classify the handwritten part and the printed part in the labeled image samples by pixels, and then remove the handwritten part from the image samples; in step S1, the semantic segmentation network is used to perform binary classification on the pixels of the handwritten part and the printed part, and the handwritten part is preliminarily removed from the image samples; the semantic segmentation network in step S1 uses Deeplab V3+, and cooperates with the backbone network resnet101 and / or Xception;

[0046] S2 Handwriting mask image processing: According to the classification results of the handwriting part and the printed part, a handwriting mask image is obtained, and then the mask image is binarized to obtain a processed filling mask image;

[0047] After the mask image is binarized in step S2, the pixel value of the handwritten part is modified so that the handwritten part is white, that is, 255, and the pixel values ​​of other non-handwritten parts are set to 0; then the hole filling algorithm is used to fill the holes in the handwritten part to obtain a processed filled mask image; the specific steps of using the hole filling algorithm are:

[0048] S21: Assuming that the original mask image is A, the original mask image A is extended outward by one to two pixels, and the pixel values ​​of the original mask image are filled with 0, that is, as the background color and marked as B;

[0049] S22: fill the pixel value of the large background of B to 255 and mark it as C;

[0050] S23: Crop the filled mask image to the original image size, remove the extended area, and mark it as D;

[0051] S24: Then invert D and add it to A to obtain the processed filling mask image; as shown in the attached figure Figure 2 As shown;

[0052] S3: Outer contour convolution denoising: by performing a sampling convolution operation on the filled mask image obtained in step S2, the outer contour of the handwritten part in the image sample is denoised;

[0053] The specific steps of step S3 are:

[0054] S31: First, perform a downsampling convolution operation on the mask image obtained in step S2. The convolution kernel size is set to 3, and the convolution matrix is ​​recorded as: The a value in the matrix is ​​non-zero;

[0055] S32: performing a dot product operation on the convolution matrix and the mask image, and the result of the dot product operation between the convolution matrix and the pixel at the zero value position in the mask image is still zero, and the result of the dot product operation between the pixel at the non-zero value position in the mask image is non-zero;

[0056] S33: Fill the pixels at the non-zero value positions to the upsampling layer according to the convolution kernel size to obtain a filled upsampling convolution image;

[0057] S34: Finally, all pixel values ​​at non-zero value positions in the upsampled convolution image are set to 255, and a matrix addition operation is performed with the image sample, thereby achieving the purpose of denoising the outer contour of the handwritten part in the image sample; as shown in the attached figure Figure 2 As shown;

[0058] S4 median filter denoising: Apply median filter to the handwritten mask image, set the grid size of the median filter to a 3*3 grid, sort the grayscale values ​​of the pixels covered in the grid in descending order, and select the median as the critical value of the filter;

[0059] S5 Adaptive threshold analysis: Through the above steps, only a small number of handwritten pixels are retained in the current image; in order to remove the residual handwritten pixels in the image sample, the image sample after filtering is subjected to adaptive threshold analysis to obtain a grayscale adaptive threshold;

[0060] The specific steps of the adaptive threshold analysis in step S5 are:

[0061] S51: First, the image matrix is ​​traversed according to the gray value range 0-255, and the distribution number of gray values ​​is counted, which is recorded as: X = {x0, x1, ..., x 255}; where x0 represents the number of pixels with a gray value of 0, and the others are similar;

[0062] S52: Secondly, the grayscale pixel probability distribution of all pixels is counted, which is denoted as: P = {p0, p1, ..., p 255}; where p0 represents the percentage of pixels with a gray value of 0, and other probability element values ​​represent the same thing (i.e. p1 represents the percentage of pixels with a gray value of 1, p 255 Indicates the percentage of pixels with a grayscale value of 255), the formula is 1-1:

[0063]

[0064] S53: Finally, for the distribution X of the gray value, a polynomial curve is fitted, as shown in formula 1-2. In order to make the curve fitting closer to the discrete points of the real distribution, the value of M is 10, and w is a parameter;

[0065]

[0066] Then find the first-order partial derivative of function f, and find the extreme point based on the monotonicity of the function.

[0067] The maximum point set is denoted as: f max ={f(x a ), f(x b ), …}; where a and b represent the first maximum value of x at point a of the function and the second maximum value of x at point b of the function respectively;

[0068] The minimum point set is denoted as: f min ={f(x j ), f(x k ), …}; where j and k represent the first minimum point of x at function point j and the second minimum point of x at function point k respectively;

[0069] According to the law of large numbers, the number of pixels with grayscale values ​​is normally distributed. In order to make the calculated adaptive grayscale value more accurate, all points above the trough of the function curve are selected as statistical participating points. For multiple troughs in the actual function curve, the maximum value of the minimum value point set is taken, which is recorded as: max(f min );

[0070] As the dividing line for grayscale pixel count statistics, calculate all pixel points corresponding to the statistical number above the dividing line, and calculate their grayscale mean as the grayscale adaptive threshold;

[0071] If the minimum point set is empty, the probability distribution statistics of grayscale pixels are used as a reference, and the probability values ​​in the probability distribution P are sorted in descending order, recorded as: P1; select the pixel points corresponding to the first half of the probability values ​​in P1, and calculate their grayscale mean as the grayscale adaptive threshold;

[0072] S6 Binarization denoising: In order to obtain an intuitive handwriting-removed image and fill in the background holes after the handwriting fonts are removed, the image needs to be binarized according to the adaptive threshold obtained in step S5; the background part of the processed image is all set to white, that is, the grayscale value is 255, and the printed part is all set to black, that is, the grayscale value is 0, thereby obtaining the final handwriting-removed image.

[0073] For ordinary technicians in this field, the specific embodiments are only illustrative descriptions of the present invention. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concepts and technical solutions of the present invention, or the concepts and technical solutions of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A handwritten font removal method based on convolution adaptive denoising, characterized in that: The specific steps include: S1 Handwritten and printed pixel classification: label several image samples, classify the handwritten and printed parts in the labeled image samples by pixels, and then remove the handwritten parts from the image samples; S2 Handwriting mask image processing: According to the classification results of the handwriting part and the printed part, a handwriting mask image is obtained, and then the mask image is binarized to obtain a processed filling mask image; S3: Outer contour convolution denoising: by performing a sampling convolution operation on the filled mask image obtained in step S2, the outer contour of the handwritten part in the image sample is denoised; S4 median filter denoising: Apply median filter to the handwritten mask image, set the grid size of the median filter, sort the grayscale values ​​of the pixels covered in the grid in descending order, and select the median as the critical value of the filter; S5 adaptive threshold analysis: in order to remove the residual handwritten pixels in the image sample, the image sample after filtering is adaptively analyzed to obtain a grayscale adaptive threshold; S6 Binarization denoising: Binarization is performed on the image according to the adaptive threshold value obtained in step S5. The background part of the processed image is completely white, that is, the gray value is 255, and the printed part is completely black, that is, the gray value is 0, so as to obtain the final handwriting-removed image; The specific steps of the adaptive threshold analysis in step S5 are: S51: First, the image matrix is ​​traversed according to the gray value range 0-255, and the distribution number of gray values ​​is counted, which is recorded as: X = {x0, x1, ..., x 255 }; where x0 represents the number of pixels with a gray value of 0, and the others are similar; S52: Secondly, the grayscale pixel probability distribution of all pixels is counted, which is denoted as: P = {p0, p1, ..., p 255 }; where P0 represents the percentage of pixels with a gray value of 0, and other probability element values ​​are expressed in the same way. The formula is 1-1: S53: Finally, for the distribution X of the gray value, a polynomial curve is fitted, as shown in formula 1-2. In order to make the curve fitting closer to the discrete points of the real distribution, the value of M is 10, and w is a parameter; Then find the first-order partial derivative of function f, and find the extreme point based on the monotonicity of the function. The maximum point set is denoted as: f max ={f(x a ), f(x b ), …}; where a and b represent the first maximum value of x at point a of the function and the second maximum value of x at point b of the function respectively; The minimum point set is denoted as: f min ={f(x j ), f(x k ), …}; where j and k represent the first minimum point of x at function point j and the second minimum point of x at function point k respectively; According to the law of large numbers, the number of pixels with grayscale values ​​is normally distributed. In order to make the calculated adaptive grayscale value more accurate, all points above the trough of the function curve are selected as statistical participating points. For multiple troughs in the actual function curve, the maximum value of the minimum value point set is taken, which is recorded as: max(f min ); As the dividing line for grayscale pixel count statistics, calculate all pixel points corresponding to the statistical number above the dividing line, and calculate their grayscale mean as the grayscale adaptive threshold; If the minimum point set is empty, the probability distribution statistics of grayscale pixels are used as a reference, and the probability values ​​in the probability distribution P are sorted in descending order, denoted as: P1; the pixel points corresponding to the first half of the probability values ​​in P1 are selected, and their grayscale mean is calculated as the grayscale adaptive threshold.

2. The handwritten font removal method based on convolution adaptive denoising according to claim 1 is characterized in that: In step S1, a semantic segmentation network is used to classify pixels of the handwritten part and the printed part respectively, and the handwritten part is preliminarily removed from the image sample.

3. The handwritten font removal method based on convolution adaptive denoising according to claim 2 is characterized in that: After the mask image is binarized in step S2, the pixel value of the handwritten part is modified to make the handwritten part white, and then the pixel value of the non-handwritten part is set, and the hole filling algorithm is used to fill the holes of the handwritten part to obtain a processed filled mask image; wherein the specific steps of using the hole filling algorithm are: S21: assuming that the original mask image is A, the original mask image A is extended outward by one to two pixels, and the pixel value of the original mask image is filled to 0, that is, as the background color and marked as B; S22: fill the pixel value of the large background of B to 255 and mark it as C; S23: Crop the filled mask image to the original image size, remove the extended area, and mark it as D; S24: Then invert D and add it to A to obtain a processed filling mask image.

4. The handwritten font removal method based on convolution adaptive denoising according to claim 2 is characterized in that: The specific steps of step S3 are: S31: First, perform a downsampling convolution operation on the mask image obtained in step S2. The convolution kernel size is set to 3, and the convolution matrix is ​​recorded as: The a value in the matrix is ​​non-zero; S32: performing a dot product operation on the convolution matrix and the mask image, and the result of the dot product operation between the convolution matrix and the pixel at the zero value position in the mask image is still zero, and the result of the dot product operation between the pixel at the non-zero value position in the mask image is non-zero; S33: Fill the pixels at the non-zero value positions to the upsampling layer according to the convolution kernel size to obtain a filled upsampling convolution image; S34: Finally, all pixel values ​​at non-zero value positions in the upsampled convolution image are set to 255, and a matrix addition operation is performed with the image sample, thereby achieving the purpose of denoising the outer contour of the handwritten part in the image sample.

5. The handwritten font removal method based on convolution adaptive denoising according to claim 2 is characterized in that: The semantic segmentation network in step S1 uses Deeplab V3+ and cooperates with the backbone network resnet101 and / or Xception.

6. The handwriting font removal method based on convolution adaptive denoising according to claim 2 is characterized in that: In step S4, the size of the value filter is set to a 3*3 grid.

Images