A method for protecting privacy data in archive form images

Abstract

The present invention discloses a method for protecting privacy data in an archive table image. The method uses the RGB color difference of a color image to fade the color seal and other contents of the table image; scales the image to a specific size, and obtains an edge strength map of the image through a convolution operation; obtains candidate horizontal and vertical line equations using Hough transform; performs a backtracking search for other possible tracking trajectories; obtains the coordinate information of the utilized row from the archive user, and automatically blurs the privacy part in the archive image and retains the part that needs to be used. The present invention has a fast calculation speed, and can complete the recognition and privacy blurring processing within 1 second on an ordinary office computer; and has a strong anti-interference ability, and can accurately recognize old archives, tables with color seals, original traces of alteration, and extremely faint table lines. In addition, the method has good compatibility, and can effectively recognize and process tables with different aspect ratios, different numbers of rows and columns, and different header styles.

Description

Technical Field

[0001] The invention relates to the field of archive utilization and image processing, in particular to an archive table image recognition and processing algorithm. Background Art

[0002] In order to respect the original records and authenticity of the contents of the archives, the archives management department will scan and preserve the original archives. When issuing archive certificates and other utilization materials, the information of others in the image that is not related to the archive users will be masked or blurred.

[0003] The current masking method is to manually use image processing software such as Paint or PhotoShop to open the original archive image, manually select the part that needs to be processed for privacy, process the image, and then save, print, sign and seal it and deliver it to the archive user. The manual selection of privacy processing is relatively cumbersome.

[0004] Computer vision technology has long been used to identify tables in document images. Traditional image recognition and processing technologies include image tilt correction, image binarization, horizontal and vertical projection of table horizontal and vertical lines, and Hough variation voting to obtain line segment equations and then perform line segment tracking detection. This type of method has a good effect when the image background is clean and the table is simple and standardized. However, a large number of archive tables have various interference factors such as complex headers, old and dark, curved lines, handwriting and signatures covering the table image, etc., making it difficult to use the above methods to obtain table wireframe information.

[0005] Another branch of computer vision technology is machine learning technology based on deep neural networks. This technology uses multi-layer convolutional layers, pooling layers, activation functions, loss functions and other methods to construct a learning mechanism that can learn a large number of samples by itself, from features such as the bottom edge of the image to features such as high-level structures. Disadvantages include: a large number of learning samples need to be prepared and labeled, and higher computing power is required for learning and reasoning calculations. Among them, the workload of collecting and labeling a large number of archive form sample images in the early stage of machine learning is very large. Even if a large number of manpower is organized to complete this work, the inference model obtained by machine learning will have a certain delay when operated on ordinary office computers in general archive utilization departments, resulting in inconvenience in application. Summary of the invention

[0006] The purpose of the present invention is to provide a method for protecting privacy data in archive table images. The method has fast calculation speed and strong anti-interference ability. It can accurately identify old archives, tables with color seals, original traces of alteration and extremely faint table lines, and has good compatibility. It can effectively identify and process tables with different aspect ratios, different numbers of rows and columns, and different header styles.

[0007] The technical solution of the present invention: a method for protecting private data in an archive table image, characterized in that it specifically comprises the following steps:

[0008] A method for protecting privacy data in an archive form image comprises the following steps:

[0009] 1) First, use the RGB color difference of the color image to fade the color stamp and other contents of the table image;

[0010] 2) Scale the image to a specific size, design a horizontal edge detection operator with good compatibility under this image size, and obtain the edge intensity map of the image through convolution operation;

[0011] 3) Using Hough transform to obtain candidate horizontal and vertical line equations;

[0012] 4) According to the straight line equation, line tracking is performed, and the line coordinates of no more than 15 pixels are memorized using a circular queue to determine the bending of the line in real time; if the line is too bent, it is judged as interference, and the coordinate information stored in the queue is used to backtrack and search for other possible tracking trajectories;

[0013] 5) Sort the Y coordinates of the horizontal lines that run through the table, and identify the header based on the fact that the header row height is significantly different from that of ordinary rows;

[0014] 6) Using a graphical interactive interface, obtain the coordinate information of the row to be used from the archive user, and combine it with the table information identified in the above steps to automatically blur the private part of the archive image and retain the part that needs to be used.

[0015] The detection operator described in step 2) is to construct an edge detection operator matrix H with n rows and m columns, the middle row value of the matrix is ​​negative, the element values ​​of the two end rows are positive, the element values ​​of each row are the same, the sum of all positive element values ​​of the matrix is ​​1, and the sum of negative element values ​​is -1, where n is an integer not greater than 7, and m is an integer not greater than 5.

[0016] The method of obtaining candidate horizontal and vertical line equations by using Hough transform in step 3) includes the following two steps:

[0017] Perform Hough transform on the image, referred to as voting: the voting parameter space is taken as a two-dimensional space: the row coordinate represents the intercept of the line, the column coordinate represents the inclination angle of the line, and the height is consistent with the document height; the voting threshold is that all edges with an edge strength of +5 are eligible for 1 vote, ensuring that weak edges and strong edges have the same number of votes when the length is the same, which is convenient for distinguishing those horizontal lines of the same length in the voting results; when the edge strength is greater than 0 and less than +5, it is considered to be a line illusion caused by the paper and the scanning instrument;

[0018] Obtain the equation parameters of the candidate lines: In the Hough variable parameter space diagram, find the maximum value as the table line width maxValH; traverse the parameter space diagram, and examine all local maxima that reach 70% of maxValH, so as to detect discontinuous and missing table lines as much as possible.

[0019] The line segment tracking algorithm in step 4) that can track appropriate bends, discontinuities and severe stroke interference is to track the line under the guidance of the straight line equation, and obtain the inclination angle and intercept of the straight line through the parametric equation of the straight line, and the intercept is the Y coordinate of the intersection with the straight line at X=0; in the edge strength diagram, starting from the point X=0, Y=intercept, move to the right with the straight line inclination angle; take t=20 as the threshold, and take the point where the maximum value of the upper, middle and lower points of the current point is located as the tracking direction; when the maximum value point intensity is greater than t, it is regarded as a line point, otherwise it is regarded as a non-line point; use a circular queue to record the coordinates of the previously tracked points; if the line length is greater than 7, calculate the bending angle of the line constructed by the last 7 points; when the angle is greater than 3°, it is regarded as a tracking error and rollback is performed; all tracked lines are recorded in the set S.

[0020] The operation of sorting the Y coordinates of the horizontal lines that run through the table and identifying the table header described in step 5) is to count the votes of the x coordinates of the left and right endpoints of all lines in S, and take the x with the largest number of votes on the left end as the left boundary L of the table, and the x with the largest number of votes on the right end as the right boundary R of the table; then traverse all the lines in S, and only keep the lines with left and right endpoints near L and R respectively, as horizontal lines that can run through the table; sort these horizontal lines by y coordinates and regard them as adjacent table horizontal lines. In the upper part of the table, judge from top to bottom: if the height of two adjacent rows is more than 20%, the previous row is regarded as the header part.

[0021] The automatic blurring of the private part in the archive image and retaining the part to be used as described in step 6) refers to obtaining the row selected by the archive user, retaining a clear image of the row and the header, performing conventional image blurring on other rows, and obtaining a final usable image for printing output.

[0022] The beneficial effects of the present invention are as follows: 1. Fast calculation speed, which can complete the recognition and privacy blurring processing within 1 second on an ordinary office computer; 2. Strong anti-interference ability, which can accurately recognize old files, tables with color seals, original traces of alteration and extremely faint table lines; 3. Good compatibility, which can effectively recognize and process tables with different aspect ratios, different numbers of rows and columns, and different header styles. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a color scan with various interference features (names have been masked);

[0024] Figure 2 It is a comparison between the ordinary color-to-grayscale conversion method and the grayscale conversion method adopted in this patent;

[0025] Figure 3 is the horizontal edge map;

[0026] Figure 4 Hough transform parameter space (left) and horizontal tracking effect (right);

[0027] Figure 5 To preserve the horizontal line that runs through the table;

[0028] Figure 6 It is a schematic diagram of the straight line tracking algorithm;

[0029] Tracking from left to right, yellow means no lines, green means correct tracking, and red means interference lines

[0030] Figure 7 An image that protects privacy information is automatically generated after the user selects the use line in the graphical interactive interface. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the embodiments, but they are not intended to limit the present invention.

[0032] Embodiment 1: A method for protecting private data in an archive form image

[0033] Step 1: Standardize the size and reduce the color of the stamp and signature

[0034] Define the original image length and width as W and H, and the scaling ratio as zoom = min (1024 / W, 1024 / H). Scale the original image to the zoomed size.

[0035] In the image after resizing, the line width can be controlled within 0.5 to 2 pixels, which is convenient for subsequent detection. The color image is separated into three channels: R, G, and B. By calculating the max of the image matrix, Gray = max(R, G, B) is obtained. Gray is the image after the color is faded, which can greatly reduce the interference of color stamps, red signatures, etc. Figure 2 Comparison shown.

[0036] Step 2: Construct a horizontal edge detection operator to perform horizontal edge detection on the image

[0037] Construct an operator matrix H with n rows and m columns for edge detection operators. The recommended values ​​for n and m are 5 and 5. The values ​​in the middle row of the matrix are negative, the values ​​of the elements in the rows at both ends are positive, the values ​​of the elements in each row are the same, the sum of all positive elements in the matrix is ​​1, and the sum of negative elements is -1. The recommended values ​​for H are:

[0038] H=[+0.100,+0.100,+0.100,+0.100,+0.100;

[0039] -0.025,-0.025,-0.025,-0.025,-0.025;

[0040] -0.150,-0.150,-0.150,-0.150,-0.150;

[0041] -0.025,-0.025,-0.025,-0.025,-0.025;

[0042] +0.100,+0.100,+0.100,+0.100,+0.100;]

[0043] Perform a 2D convolution on Gray and H to obtain the horizontal edge intensity map E = conv2d(Gray,H); the effect is as follows Figure 3 shown.

[0044] The conv2d here is the 2d convolution of the convolutional neural network.

[0045] Step 3: Extract table rows

[0046] Data related to file privacy is generally in units of behavior, so here we use horizontal line tracking as an example.

[0047] Voting. Perform Hough transform (voting) on ​​the image, and take the voting parameter space as a two-dimensional space: the row coordinate represents the intercept of the line, and the column coordinate represents the inclination angle of the line (the angle resolution is 0.1°, and the angle range is ±3°). There are 61 columns in total. If necessary, you can expand this range by yourself, and the height is consistent with the document height. The voting threshold is that all edges with an edge strength of +5 are eligible for 1 vote, ensuring that weak edges and strong edges have the same number of votes when the length is the same, so that those horizontal lines of the same length in the table can be distinguished in the voting results. When the edge strength is greater than 0 and less than +5, it is considered to be a line artifact caused by the paper and the scanning instrument.

[0048] Get the equation parameters of the candidate line. In the Hough variable parameter space diagram ( Figure 4 The black part on the left) is obtained, and the maximum value is the table line width maxValH. The parameter space diagram is traversed, and all local maxima that reach 70% of maxValH are examined to detect as many discontinuous and missing table lines as possible.

[0049] Step 4: Using the equation of the line as a guide, trace the line ( Figure 6As shown). The parametric equation of the line can be used to obtain the inclination angle and intercept of the line (the Y coordinate of the intersection with the line at X=0). In the edge strength graph, starting from the point X=0, Y=intercept, move to the right with the inclination angle of the line. Take t=20 as the threshold, and take the point with the maximum value of the top, middle and bottom points of the current point as the tracking direction. When the maximum point strength is greater than t, it is considered a line point, otherwise it is considered a non-line point.

[0050] Use a circular queue (a data structure) to record the coordinates of the previously tracked points. If the line length is greater than 7, calculate the bending angle of the line constructed by the last 7 points. When the angle is greater than 3°, it is considered a tracking error and backed off. All tracked lines are recorded in the set S.

[0051] The reason for using a circular queue is that the curvature of the tracking line is examined only in the seven points tracked recently, and the coordinate storage space of points in a longer range can be recycled.

[0052] Step 5: Obtain the horizontal penetration line and header of the table

[0053] Count the votes of the x coordinates of the left and right endpoints of all lines in S, and take the x with the largest number of votes on the left end as the left boundary L of the table, and the x with the largest number of votes on the right end as the right boundary R of the table.

[0054] Then traverse all the lines in S and keep only the lines whose left and right endpoints are near L and R respectively, which are considered as horizontal lines that can penetrate the table. Figure 6 shown.

[0055] These horizontal lines are sorted by y coordinates and regarded as adjacent table horizontal lines. In the upper part of the table, judging from top to bottom: if the height of two adjacent rows is more than 20% higher than that of the previous row, the previous row is regarded as the header part.

[0056] Step 6: Apply privacy protection

[0057] From the software interactive interface, obtain the row selected by the file user, keep the clear image of the row and the header, and perform conventional image blurring on other rows to obtain the final usable image for printing output. Figure 7 shown.

Claims

1. A method for protecting privacy data in an archive form image, characterized in that: The specific steps include: 1) First, use the RGB color difference of the color image to fade the color stamp and other contents of the table image; 2) Scale the image to a specific size, design a horizontal edge detection operator with good compatibility under this image size, and obtain the edge intensity map of the image through convolution operation; 3) Use Hough transform to obtain candidate horizontal and vertical line equations; The method of obtaining candidate horizontal and vertical line equations by using Hough transform in step 3) includes the following two steps: Perform Hough transform on the image, referred to as voting: the voting parameter space is taken as a two-dimensional space: the row coordinate represents the intercept of the line, the column coordinate represents the inclination angle of the line, and the height is consistent with the document height; the voting threshold is that all edges with an edge strength of +5 are eligible for 1 vote, ensuring that weak edges and strong edges have the same number of votes when the length is the same, which is convenient for distinguishing those horizontal lines of the same length in the voting results; when the edge strength is greater than 0 and less than +5, it is considered to be a line illusion caused by the paper and the scanning instrument; Obtain the equation parameters of the candidate line: in the Hough variable parameter space diagram, find the maximum value as the table line width maxValH; traverse the parameter space diagram, and examine all local maxima that reach 70% of maxValH, so as to detect the discontinuous missing table lines as much as possible; 4) According to the straight line equation, line tracking is performed, and the line coordinates of no more than 15 pixels are memorized using a circular queue to determine the bending of the line in real time; if the line is too bent, it is judged as interference, and the coordinate information stored in the queue is used to backtrack and search for other possible tracking trajectories; 5) Sort the Y coordinates of the horizontal lines that run through the table, and identify the header based on the fact that the header row height is significantly different from that of ordinary rows; 6) Using a graphical interactive interface, obtain the coordinate information of the used row from the archive user, and combine it with the table information identified in the above steps to automatically blur the private part of the archive image and retain the part that needs to be used.

2. The method for protecting private data in an archive form image according to claim 1, characterized in that: The detection operator described in step 2) is to construct an edge detection operator matrix H with n rows and m columns, where the middle row value of the matrix is ​​negative, the element values ​​of the two end rows are positive, the element values ​​of each row are the same, the sum of all positive element values ​​of the matrix is ​​1, and the sum of negative element values ​​is -1, where n is an integer not greater than 7, and m is an integer not greater than 5.

3. The method for protecting privacy data in an archive form image according to claim 1, characterized in that: The line segment tracking algorithm in step 4) that can track appropriate bends, discontinuities, and severe stroke interference is to track the line under the guidance of the line equation, and obtain the inclination angle and intercept of the line through the parametric equation of the line, and the intercept is the Y coordinate of the intersection with the line at X=0; In the edge strength diagram, start from the point X=0, Y=intercept, and move to the right with a straight line inclination angle; take t=20 as the threshold, and take the point with the maximum value of the upper, middle and lower points of the current point as the tracking direction; when the maximum point strength is greater than t, it is regarded as a line point, otherwise it is regarded as a non-line point; use a circular queue to record the coordinates of the previously tracked points; if the line length is greater than 7, calculate the bending angle of the line constructed by the nearest 7 points; when the angle is greater than 3°, it is regarded as a tracking error and backs off; record all tracked lines in the set S.

4. The method for protecting private data in an archive form image according to claim 1, characterized in that: The operation of sorting the Y coordinates of the horizontal lines that run through the table and identifying the table header as described in step 5) is to count the votes of the x coordinates of the left and right endpoints of all lines in S, and take the x with the largest number of votes on the left end as the left boundary L of the table, and the x with the largest number of votes on the right end as the right boundary R of the table; then traverse all the lines in S, and only keep the lines whose left and right endpoints are near L and R respectively, as horizontal lines that can run through the table; sort these horizontal lines by y coordinates and regard them as adjacent table horizontal lines; in the upper part of the table, judge from top to bottom: if the height of two adjacent rows exceeds 20%, the upper row is regarded as the header part.

5. The method for protecting private data in an archive form image according to claim 1, characterized in that: The automatic blurring of the private part in the archive image and retaining the part to be used described in step 6) refers to obtaining the row selected by the archive user, retaining a clear image of the row and the header, performing conventional image blurring on other rows, and obtaining a final usable image for printing output.

Images