A paper hydrological yearbook table structure identification method, device and storage medium

Abstract

The application provides a paper hydrological yearbook table structure identification method, comprising the following steps: obtaining a scanned file of a paper hydrological yearbook and converting it into a picture file; obtaining a table image and attribute information thereof in each converted picture file; performing image preprocessing, correction and super-resolution conversion on the obtained table image; performing straight line detection on the converted table image to generate a table cell; performing merged cell and multi-layer nested structure identification on the generated table cell; identifying the text in the table image, obtaining the bounding box of each text region, and matching the obtained bounding box of the text region with its nearest neighbor cell; converting the coordinates of the cell after text matching into row and column serial numbers, mapping to an Excel table form, and filling the text recognition result into the Excel table cell; performing general verification and hydrological yearbook data compliance verification on the filled Excel table, and taking the verification as the final identification result after passing the verification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and table recognition, and in particular to a paper hydrological yearbook table structure recognition method, device and storage medium. BACKGROUND

[0002] As a valuable carrier of hydrological test data, the hydrological yearbook carries rich information of long-term evolution of nature and profound imprint of human activity impact, and reflects the changes of natural environment and the influence of human activity. However, the traditional paper version of the hydrological yearbook has many inconveniences in preservation and utilization, such as easy damage, difficult retrieval and analysis, etc. Digitizing the paper hydrological yearbook table can greatly improve the efficiency of data retrieval and use. The traditional paper data needs to be manually reviewed, which is time-consuming and prone to errors, while the digitized data can be quickly retrieved by computer, providing instant information support for researchers and decision makers. In addition, digitized hydrological data is convenient for large-scale data analysis and model construction, which helps to deeply mine the scientific laws behind the data and predict future hydrological trends.

[0003] However, the automatic recognition of the table of the hydrological yearbook faces multiple difficulties compared to ordinary tables:

[0004] 1) The size of the table of the hydrological yearbook is usually large, reaching more than 1000x2000 pixels, and the aspect ratio is not fixed, which brings challenges to image processing and recognition.

[0005] 2) The table of the hydrological yearbook image has a large number of cells and dense text information, requiring the recognition algorithm to have high accuracy and fineness. However, there is often not enough table line in the table image, which makes it difficult to reconstruct the table structure. In addition, there are a large number of blank cells and blank columns, and there is no table line as a reference, which further increases the difficulty of table reconstruction and text positioning. More complex is that the hydrological yearbook table image may contain two or more tables, each table region has a title, which destroys the overall structure of the table and increases the complexity of recognition. At the same time, the structure of the hydrological yearbook table itself may be very complex, including merged cells, divided sub-tables, and sub-headings in the table content, etc., which requires the recognition algorithm to be flexible to deal with various complex situations.

[0006] 3) The image quality of the hydrological yearbook table is also a problem that cannot be ignored. The table lines may be curved, the table angles may be not right, and the text may be blurred, which brings additional challenges to accurate table recognition. SUMMARY

[0007] In view of the difficulty of automatically identifying paper hydrological yearbook table images, the paper hydrological yearbook table structured identification method is provided, which comprehensively utilizes image processing technology, target detection technology, character recognition technology, table structured identification technology and the like, and performs structured identification on complex paper hydrological yearbook table images, can process various table types in paper hydrological yearbooks, and can overcome the problem of low table image quality caused by poor quality of hydrological paper yearbooks, scanning and the like.

[0008] To achieve the above object, the application adopts the following technical scheme:

[0009] The first aspect of the application discloses a paper hydrological yearbook table structured identification method, comprising the following steps:

[0010] Obtain the scanning file of the paper hydrological yearbook and convert it into a picture file;

[0011] Obtain the table image and attribute information in each converted picture file, wherein the attribute information at least includes title character information and page number information of the table image;

[0012] Perform image preprocessing on the obtained table image, and perform correction and super-resolution conversion;

[0013] Perform straight line detection on the converted table image, and generate a table cell based on the straight line detection result;

[0014] Perform merged cell and multi-layer nested structure identification on the generated table cell;

[0015] Identify the characters in the table image based on the OCR technology, obtain the boundary box of each character area, and match the obtained boundary box of the character area with the nearest neighbor cell;

[0016] Convert the coordinates of the cell after character matching into row and column serial numbers, map to the Excel table form, and fill the character recognition result into the Excel table cell;

[0017] Perform general verification and hydrological yearbook data compliance verification on the filled Excel table, and take the verification as the final identification result after passing the verification.

[0018] Further, the scanning file of the paper hydrological yearbook is converted into a picture file, which comprises:

[0019] Obtain the PDF scanning file of the paper hydrological yearbook through on-site scanning or reading;

[0020] Parse the PDF scanning file, obtain the size information, and judge the file format based on the size information;

[0021] Render the PDF scanned file into a single picture file based on the file format and preset picture parameters.

[0022] Further, the obtaining of the table image in each converted picture file and the attribute information thereof comprises:

[0023] Detecting the position of the table in the picture based on the pre-trained table region detection model to obtain the table image;

[0024] Based on the pre-trained title detection model, the title picture is cropped from the picture, and the text in the title picture is recognized by the text recognition algorithm to obtain the title text information;

[0025] Based on the pre-trained page number detection model, the page number picture is cropped from the picture, and the text in the page number picture is recognized by the text recognition algorithm to obtain the page number information;

[0026] The title text information and the page number information are associated with the table image as the attribute information of the table.

[0027] Further, the image preprocessing, correction and super-resolution conversion of the obtained table image comprise:

[0028] The image preprocessing of the table image comprises image graying, image filtering and image binarization;

[0029] The preprocessed table image is subjected to angle correction and distortion correction;

[0030] The super-resolution of the corrected table image is realized based on the pre-trained convolutional neural network.

[0031] Further, the straight line detection on the converted table image and the generation of table cells based on the straight line detection result comprise:

[0032] Adding table frame lines to the positions of missing table lines on the table image;

[0033] The original table image is preprocessed by using a morphological dilation algorithm to make the dotted lines and dotted lines in the table image continuous to form complete lines, and straight line detection is performed on the preprocessed image;

[0034] The detected straight lines are merged by using a straight line merging strategy based on distance and angle to obtain table lines;

[0035] Based on the detected table lines, the intersection points of all table lines are calculated, and the cells are generated based on the intersection points.

[0036] Further, the adding of table frame lines to the positions of missing table lines on the table image comprises:

[0037] Detecting non-text lines in the table image, traversing all lines of the table image from the first line, calculating the distance between non-blank pixels in each line; for a certain line, if there are pixel pairs with distance within a preset distance range and not adjacent, and the number of such pixel pairs exceeds a preset number threshold, the line is determined as a text line, otherwise it is a non-text line;

[0038] Applying morphological closing operation to remove black dots or black line segments of non-text lines; calculating connected regions by OpenCV connected region operator, obtaining height information of the connected regions, and sorting the connected regions according to the height from large to small;

[0039] For text lines, calculate connected regions by OpenCV connected region operator, obtain the height of each connected region, remove the maximum height of the front preset proportion and the minimum height of the rear preset proportion, and calculate the average height of the remaining connected regions as the text line height;

[0040] Compare the connected regions of non-text lines one by one with the text line height. If the height is less than the text line height, draw a straight line at half the height of the non-text line connected region, otherwise do not draw a line, and obtain the table image with added table frame lines.

[0041] Further, the straight line merging strategy based on distance and angle is used to merge the detected straight lines to obtain table lines, which includes:

[0042] Traverse all detected straight lines, and for each pair of straight lines, calculate the minimum vertical distance and angle difference between them;

[0043] If the distance between the two straight lines is less than a preset threshold and the angle difference is also less than a preset threshold, it is determined that the two straight lines are actually one, and they are merged into a new straight line;

[0044] Place the merged straight line at the center position of the original two straight lines, and keep the original length and direction characteristics.

[0045] Further, the merging cell recognition of the generated table cells includes:

[0046] For horizontal table lines, check whether there are multiple intersection points with the same x coordinate in the vertical direction; for vertical table lines, check whether there are multiple intersection points with the same y coordinate in the horizontal direction;

[0047] If multiple intersection points meeting the conditions are detected, it is determined that there are merged cells across rows or columns;

[0048] For the merged cell across rows, find the minimum and maximum y coordinates of the multiple intersection points that meet the conditions, and take them as the upper and lower boundaries of the cell; for the merged cell across columns, find the minimum and maximum x coordinates of the multiple intersection points that meet the conditions, and take them as the left and right boundaries of the cell.

[0049] Further, the identification of the generated table cells in a multi-layer nested structure includes:

[0050] First, identify the outermost closed area as the starting point of the nested structure, and record the four corner point coordinates of the outermost closed area;

[0051] Further detect the internal table lines and intersection points in the identified outermost closed area, and recursively call the closed area detection algorithm to analyze the internal nested structure layer by layer;

[0052] When it is detected that a certain closed area no longer contains any table lines or intersection points, the recursion terminates, and the closed area is the bottom layer of a single cell.

[0053] Further, the conversion of the coordinates of the cells matched by the text into row and column numbers and the mapping to the Excel table form includes:

[0054] Define a reference frame, select the first valid cell in the top left corner of the table as the origin, and then determine the row and column of other cells according to the relative position between each horizontal line and vertical line;

[0055] Traverse all detected horizontal and vertical lines, respectively, and count the number to determine the total number of rows and columns in the entire table;

[0056] For each cell, calculate which row and column it belongs to according to the position of its four corner points relative to the reference frame;

[0057] Based on the calculated number of rows and columns, create a new Excel worksheet object to reconstruct the table.

[0058] Further, the traversal of all detected horizontal and vertical lines further includes filtering the detected straight lines based on the following rules:

[0059] Length-based filtering: set a minimum length threshold to filter out lines with a length less than the threshold;

[0060] Intersection-based filtering: only keep lines that intersect with other lines;

[0061] Range-based filtering: only keep lines within the table boundary.

[0062] Further, the hydrological yearbook data compliance verification on the filled Excel table comprises multiple or all of the following verifications:

[0063] For the daily precipitation table, in the numerical area, it is verified whether all cells are legal characters, in the statistical value area, it is verified whether the precipitation is consistent with the corresponding month, whether the precipitation days are correct, whether the maximum daily amount is consistent with the numerical area, and whether the sum of the annual statistics and the numerical area is consistent.

[0064] For the maximum precipitation table of each period, it is verified whether all characters are legal characters, whether the station name exists, and whether the long-period precipitation is greater than or equal to the short-period maximum precipitation.

[0065] The second aspect of the present application also discloses a computer device comprising a memory and a processor, the memory stores computer instructions, and the processor executes the computer instructions to perform the paper hydrological yearbook table structure identification method of the first aspect.

[0066] The third aspect of the present application also discloses a computer readable storage medium, which stores computer instructions, and the computer instructions are executed by a computer to perform the paper hydrological yearbook table structure identification method of the first aspect.

[0067] The beneficial technical effects of the present application are as follows:

[0068] 1) The paper hydrological yearbook table structure identification method of the present application can realize the full-process automatic processing of the hydrological yearbook table from the paper scanned PDF file to the structured identification, and provides an effective technical means for the digital processing of the paper hydrological yearbook table.

[0069] 2) The paper hydrological yearbook table structure identification method of the present application has a wide range of adaptation and can process all types of tables in the paper hydrological yearbook.

[0070] 3) The paper hydrological yearbook table structure identification method of the present application can achieve good recognition effect for the table image with poor quality caused by poor quality of the paper yearbook or scanning factors, through table image preprocessing, angle and distortion correction and super-resolution conversion.

[0071] 4) The paper hydrological yearbook table structure identification method of the present application can automatically judge whether there is a missing table frame line, and can automatically add a table line at the position of the missing table frame line to form a framed table, thereby reducing the recognition difficulty of the unframed table, and the hydrological yearbook table with missing table lines can also be recognized with high accuracy.

[0072] 5) The paper hydrological yearbook table structured identification method of the present application optimizes the common merged cells and the complex nested structure of the cells in the hydrological yearbook table, can effectively identify the merged cells through collinearity detection, and gradually restores the nested cell structure through recursive inner cell, so that the table structure can be accurately reconstructed.

[0073] 6) The paper hydrological yearbook table structured identification method of the present application can comprehensively check the automatic identification result through general checksum and hydrological yearbook data compliance check, not only can repair unreasonable table structure, but also effectively ensure the accuracy of the identification result, and greatly reduce the time and labor cost of manual cell-by-cell check. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 The flowchart of the paper hydrological yearbook table structured identification method embodiment of the present application is shown. DETAILED DESCRIPTION

[0075] Embodiments of the present application will be described in more detail by referring to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes, and are not intended to limit the scope of protection of the present application.

[0076] Referring to Figure 1 The paper hydrological yearbook table structured identification method of the present application comprises the following steps:

[0077] S1, obtaining the scanned file of the paper hydrological yearbook and converting it into a picture file.

[0078] As a preferred embodiment, the step is implemented by the following process in the present embodiment:

[0079] S11, obtaining the PDF scanned file of the paper hydrological yearbook by on-site scanning or reading.

[0080] Specifically, the way to obtain the PDF scanned file of the paper hydrological yearbook can be on-site scanning or reading the stored scanned PDF scanned file, which is not limited by the present application.

[0081] S12, analyzing the PDF scanned file to obtain size information and judging the file format based on the size information.

[0082] Specifically, the PDF file needs to be analyzed to obtain the width and height of the PDF file.

[0083] The PDF file scanned from the paper hydrological yearbook usually has two formats, A3 and A4. The A3 format contains the left and right pages of the paper, and the A4 format contains only a single page. If the width of the PDF file is greater than the height, it means that the PDF file is in the A3 format, and in the subsequent rendering step, one page of the PDF file needs to be rendered into two pictures, each corresponding to a single page of the paper. If the height of the PDF is greater than the width, it means that the PDF file is in the A4 format, and in the subsequent rendering step, one page of the PDF file is rendered into one picture, which is also a single page of the paper.

[0084] S13, based on the file format and the preset image parameters, rendering the PDF scanned file into a single image file.

[0085] Specifically, if a hydrological paper yearbook has 100 pages, that is, 50 sheets of paper, the final rendering result is 100 pictures. In the A4 format PDF file, it is a 100-page PDF file, which is rendered into 100 pictures. In the A3 format PDF file, it is a 50-page PDF file, and one page of the A3 format PDF file needs to be rendered into two pictures, that is, 50-page PDF file is rendered into 100 pictures.

[0086] When rendering each page of the PDF file into an image, the following parameters need to be set:

[0087] ① Image format: the conventional formats are jpg and png;

[0088] ② Image size: such as width 2480 pixels, height 3509 pixels;

[0089] ③ Image DPI resolution: the conventional DPI is 100 / 200 / 300;

[0090] ④ Color space: the conventional ones are RGB and CMYK space, and RGB space is set here;

[0091] ⑤ Image channel number: the conventional one is three channels.

[0092] Among them, the image size and DPI resolution are two optional settings. If the image size is set, the DPI resolution does not need to be set. If the DPI is set, the image size does not need to be set.

[0093] S2, obtaining the table image and its attribute information in each converted image file, wherein the attribute information at least includes the title text information and the page number information of the table image.

[0094] Generally, a page in a hydrological yearbook usually has one or more tables, each of which has a title, and each page has a page number.

[0095] As a preferred embodiment, in this embodiment, this step is implemented using the following flow:

[0096] S21, detecting the position of the table in the picture based on the pre-trained table region detection model, to obtain a table image.

[0097] In one example, the table region detection model used by the present application is trained in the following manner:

[0098] 1) Construct a data set for table region detection, which is used to train the table region detection model. The data set is composed of rendered pictures and labels, and the number is not limited, the more the better, usually more than 100 images. It is generally considered that the upper left corner of the picture is the starting origin (0, 0), and x and y represent the position of any pixel point in the image, x is the width direction and y is the height direction. The table region in the picture can be represented by a rectangle, and there are many ways to represent the rectangle, for example, two kinds are given here: ① the coordinates of the upper left corner and the lower right corner of the rectangle, which can form a rectangle; ② the coordinates of the upper left corner, the width and the height of the rectangle. The label is used to record the position information of each table region. If there is a table in a picture, the label recorded by the above method ① is the coordinates of the upper left corner (x1, y1) and the coordinates of the lower right corner (x2, y2). The label can be in many forms, which can be saved in txt format, xml format or json format.

[0099] 2) Train the table region detection model, which can be based on neural network models such as Faster R-CNN, YOLO or DETR. Set the parameters required by the model, for example:

[0100] ① Number of iterations (Epochs): how many times the model is trained on the data set;

[0101] ② Batch size: how many pictures are input at a time during training;

[0102] ③ Learning rate: determines the step size of updating the model parameters at each iteration;

[0103] ④ Loss function: measures the difference between the model's predicted value and the true value;

[0104] ⑤ Optimizer: how to adjust its internal parameters to reduce errors according to the results of the loss function;

[0105] ⑥ Weight initialization: initialization method of model parameters.

[0106] After setting the parameters, the model is trained according to the set parameters. Before training, an initialization method is selected to initialize the weights. Training is performed in small batches, with Batchsize pictures being input at a time. When all pictures in the data set are traversed, one iteration is completed. During training, the model is input with the picture X and the label Y true. The model judges the position Y pred of the table in the picture X. The loss function calculates the difference value by judging the difference between Y true and Y pred. The optimizer uses the learning force and the calculated difference value to update the weights of the model.

[0107] After training, the trained table region detection model can be used to detect the position of the table in each picture to be detected. If the label in the data set is represented in mode ①, the result of the model detection is also represented by the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2). By (x1, y1) and (x2, y2), the table region is cropped from the picture page, and a table image is obtained.

[0108] S22, based on the pre-trained title detection model, the title picture is cropped from the picture, and the text recognition algorithm is used to recognize the text in the title picture to obtain the title text information.

[0109] Specifically, the training of the title detection model can refer to the training method of the table region detection model described above, except that the label in the data set represents the position information of the title in the picture.

[0110] After training, the trained title detection model can be used to crop the title picture from the picture to be detected, and then use the OCR recognition algorithm to recognize the text in the title picture to obtain the text information of the title.

[0111] S23, based on the pre-trained page number detection model, the page number picture is cropped from the picture, and the text recognition algorithm is used to recognize the text in the page number picture to obtain the page number information.

[0112] Specifically, the training of the page number detection model can also refer to the training method of the table region detection model described above, which is not repeated here.

[0113] After training, the trained page number detection model can be used to crop the page number picture from the picture to be detected, and then use the OCR recognition algorithm to recognize the text in the page number picture to obtain the page number information.

[0114] S24, the title text information and the page number information are associated with the corresponding table image as the attribute information of the table.

[0115] It should be noted that there is no sequential order between the above steps S21, S22 and S23, and the detection order can be adjusted according to the actual situation during implementation, which does not affect the implementation of the present application scheme.

[0116] S3, image preprocessing is performed on the obtained table image, and correction and super-resolution conversion are performed.

[0117] As a preferred embodiment, in this embodiment, this step is implemented by using the following flow:

[0118] S31, image preprocessing is performed on the table image, including image graying, image filtering and image binarization. All table images need to be preprocessed, and the main purpose is to enhance the features in the table image.

[0119] Specifically, image graying is the process of converting a color image into a grayscale image. Each pixel in a grayscale image has only one brightness value, without color information. The average method is used to convert the RGB value of a color image into a grayscale image. Different weights are assigned to the three components of RGB, and then the weighted sum is calculated as the grayscale value. For example, the commonly used weights are 0.299 (red), 0.587 (green) and 0.114 (blue).

[0120] Image filtering is a process of suppressing noise in the target image while preserving the details of the image as much as possible to improve image quality. Select an appropriate filter, such as a mean filter, a Gaussian filter, a median filter, etc., and set the corresponding parameters, such as the size of the filter, the standard deviation, etc. Try different filters and corresponding parameters several times, select the best filter and parameters with the best effect, and obtain the filtered image.

[0121] Image binarization is the process of converting an image into an image containing only two pixel values. The adaptive threshold method is used to dynamically select the threshold value based on the local features of the image to obtain the binarized image.

[0122] S32, angle correction and distortion correction are performed on the preprocessed table image.

[0123] Due to improper storage of paper hydrological yearbooks or distortion during scanning, the table image may have a certain degree of rotation, or the table may be distorted. To solve this problem, the present application adopts the method of first correcting the angle of the table and then correcting the distortion.

[0124] Specifically, the process of angle correction is as follows: the edge detection operator or Hough transform is used to find the main lines in the image, the direction of the main lines is calculated to obtain the angle of rotation of the image, and the image rotation transformation is used to correct the angle of the image.

[0125] The process of distortion correction is as follows: using an edge detection operator to detect continuous line segments, and calculating the coordinates of the cross points of each continuous line segment with other line segments, calculating the coordinates of each cross point in the non-distortion case through the average of the coordinates of the cross points on the same continuous line segment on the horizontal / vertical axis, and applying an image projection transformation algorithm to map the distorted image to a non-distorted image.

[0126] S33, based on the pre-trained convolutional neural network, realizing super-resolution of the corrected table image.

[0127] In view of the compact characters, dense text information and unclear characters in the paper hydrological yearbook, the deep learning super-resolution algorithm is applied for repair. Specifically, the table image size is enlarged to 2 times the original image size, and the mapping relationship from the low-resolution image to the high-resolution image is learned through the convolutional neural network, the learned features of the low-resolution image are converted into the features of the high-resolution image by using the pixel recombination technology through deep convolution, so that the super-resolution of the image is realized.

[0128] S4, performing straight line detection on the converted table image, and generating a table cell based on the straight line detection result.

[0129] As a preferred embodiment, in this embodiment, this step is realized by using the following flow:

[0130] S41, adding a table frame line to the position of the missing table line on the table image.

[0131] In view of the problem of a large number of missing table lines in the table of the paper hydrological yearbook, the present application solves the problem by automatically adding a table frame line to the position of the missing table line on the table image. The specific process is as follows:

[0132] Detecting non-text rows in the table image. Starting from the first row, all rows of the table image are traversed, and the distance between non-blank pixels in each row is calculated. For a certain row, if there are pixel pairs with a distance within a preset distance range (such as a pixel distance of 3-20) and not adjacent, and the number of such pixel pairs exceeds a preset number threshold (such as more than 3), the row is determined as a text row, otherwise it is a non-text row.

[0133] Applying morphological closing operation to remove black points or black line segments in non-text rows. The connected region operator of OpenCV is used to calculate the connected region, and the height information of the connected region is obtained. The connected regions are sorted in descending order of height.

[0134] For the text row, the connected region operator of OpenCV is used to calculate the connected region, and the height of each connected region is obtained. The top 5% of the maximum line height and the bottom 5% of the minimum line height are removed, and the average height of the remaining connected regions is calculated as the text line height.

[0135] The connected regions of non-text lines are compared with the height of the text line one by one. If the height is less than the height of the text line, a straight line is drawn at half the height of the connected region of the non-text line, otherwise no line is drawn. The table image with added lines is obtained.

[0136] S42, a morphological dilation algorithm is used to preprocess the original table image, and straight line detection is performed in the preprocessed image.

[0137] In order to improve the continuity and detectability of the table lines, a morphological dilation algorithm is used to preprocess the original table image in this embodiment. The dilation operation is a basic morphological operation on binary images, which connects the disconnected parts by expanding the foreground objects (i.e. white pixels), so that the dotted lines and dotted lines become continuous, forming complete lines. The specific implementation process is as follows:

[0138] (1) The table image to be processed is converted into a binary image, in which the table lines are represented as white and the background is black;

[0139] (2) A suitable structure element is used to perform dilation operation to ensure that the dotted lines and dotted lines can be effectively connected, but not to the extent that the distance between adjacent lines is too small;

[0140] (3) The dilated image is cleaned as necessary, such as removing isolated small dots or noise, to ensure the quality of subsequent processing.

[0141] After preprocessing, the next step is to detect the straight lines in the image. In this embodiment, Hough Transform is used for straight line detection, which is a classic feature extraction technique, especially suitable for detecting regular shapes such as straight lines. Hough Transform can map points in image space to curves in parameter space, thereby realizing straight line detection. The specific implementation scheme is as follows:

[0142] (1) Apply Hough Transform algorithm to analyze the dilated image to find possible straight lines;

[0143] (2) According to the set threshold, filter out the significant straight line segments, which represent the actual boundaries of the table;

[0144] (3) Calculate the position and direction parameters of each straight line for further processing.

[0145] S43, a straight line merging strategy based on distance and angle is used to merge the detected straight lines to obtain the table lines.

[0146] Considering that there can be multiple parallel lines very close to each other in the real table, which can be caused by printing quality, scanning error, etc. To simplify the table structure and reduce redundant information, the invention uses a distance and angle based line merging strategy:

[0147] (1) Traverse all detected lines, for each pair of lines, calculate the minimum perpendicular distance and angle difference between them;

[0148] (2) If the distance between two lines is less than a preset threshold and the angle difference is also less than a preset threshold, it is judged that the two lines are actually one, and they are merged into a new line;

[0149] (3) The merged line should be located at the center position of the original two lines, and the original length and direction characteristics should be maintained.

[0150] S44, based on the detected table lines, calculate the intersection points of all table lines, and generate cells based on the intersection points.

[0151] Based on table line detection, the intersection points of all table lines are calculated, and the intersection points are used to form cells. In this embodiment, the following method is used to realize it:

[0152] First, all detected lines need to be parameterized. Usually, a straight line can be represented by slope-intercept form (y = mx + b) or general form (Ax + By + C = 0). If two lines are not parallel, they must intersect at a point.

[0153] It should be noted that not all calculated intersection points are valid, because some intersection points can be caused by image noise or false detection. Therefore, some rules need to be set to filter out these invalid intersection points according to the actual situation, for example: only keep the intersection points located within the image boundary; exclude new intersection points that are very close to other known intersection points to avoid repeated counting.

[0154] Once all valid intersection points are obtained, an adjacency graph can be constructed to represent the connection between adjacent intersection points. In this graph, each node represents an intersection point, and the edge represents that two intersection points are connected by the same line. By traversing the adjacency graph, all closed regions are found, that is, a group of spaces completely surrounded by line segments. These closed regions are the cells in the table. For each possible cell, record the position information of its four corner points for subsequent processing.

[0155] S5, merging cells and identifying multi-layer nested structure for the generated table cells.

[0156] For some complex table layout, especially the hydrological yearbook table targeted by the present application, there are many cases such as merged cells or multi-layer nested structure, and simple closed area detection may not be enough to correctly segment all cells. At this time, the following strategies can be used:

[0157] (1) For merged cells spanning multiple rows or columns, check whether multiple intersection points are collinear, and adjust the cell boundaries accordingly. When multiple intersection points are detected to be collinear, it can be inferred that there is a merged cell. For example, if a cell spans multiple rows, there will be multiple intersection points in the vertical direction that lie on the same straight line. At this time, the range of the merged cell can be correctly identified by adjusting the boundaries of the cell.

[0158] As a preferred embodiment, in this embodiment, the generated table cells are subjected to merged cell recognition, which includes:

[0159] For horizontal table lines, check whether multiple intersection points are collinear in the vertical direction, i.e. have the same x coordinate; for vertical table lines, check whether multiple intersection points are collinear in the horizontal direction, i.e. have the same y coordinate;

[0160] If multiple intersection points that meet the collinear condition are detected, it is determined that there is a merged cell spanning rows or columns;

[0161] For merged cells spanning rows, find the minimum and maximum y coordinates of multiple intersection points that meet the collinear condition, and use them as the upper and lower boundaries of the cell; for merged cells spanning columns, find the minimum and maximum x coordinates of multiple intersection points that meet the collinear condition, and use them as the left and right boundaries of the cell.

[0162] (2) For multi-layer nested table structure, a recursive method can be used for analysis. First, identify the outermost closed area, and then further search for the internal closed area within this area. By recursively analyzing each layer of closed area, until the single cell at the bottom layer is identified.

[0163] As a preferred embodiment, in this embodiment, the generated table cells are subjected to multi-layer nested structure recognition, which includes:

[0164] First, identify the outermost closed area as the starting point of the nested structure, and record the coordinates of the four corner points of the outermost closed area;

[0165] Further detect the internal table lines and intersection points within the identified outermost closed area, and recursively call the closed area detection algorithm to analyze the internal nested structure layer by layer;

[0166] When it is detected that a certain enclosed area no longer contains any table lines or intersections, the recursion terminates, and the enclosed area is the bottommost single cell.

[0167] S6、Based on the OCR technology, the text in the table image is recognized, the boundary box of each text area is obtained, and the boundary box of the obtained text area is matched with its nearest neighbor cell.

[0168] As a preferred embodiment, in this embodiment, this step is implemented using the following flow:

[0169] The text in the table image is recognized using OCR (Optical Character Recognition) technology, and the accurate position of each text in the image (i.e., the boundary box) is obtained;

[0170] The boundary box of each text area is matched with its nearest neighbor cell. In this embodiment, the principle of minimizing the distance between the geometric center is used: for each text boundary box, the cell closest to its geometric center is found as the corresponding container.

[0171] S7、The coordinates of the cell after text matching are converted into row and column numbers, mapped to an Excel table form, and the text recognition result is filled into the Excel table cell.

[0172] As a preferred embodiment, in this embodiment, this step is implemented using the following flow:

[0173] In order to conveniently convert the cell coordinates into row and column numbers, a reference frame is first defined. Usually, the first valid cell at the top left corner of the table is selected as the origin (0, 0), and then the row and column of other cells are determined according to the relative position between the horizontal lines and the vertical lines. All detected horizontal lines and vertical lines are traversed, and their number is counted, and the number of rows and columns in the entire table is determined based on this. For each cell, according to the position of its four corner points relative to the reference frame, it is calculated that it belongs to which row and which column. Specifically, the row number can be determined by comparing the position relationship between the upper and lower boundaries of the cell and the horizontal lines; similarly, the column number can be determined by comparing the position relationship between the left and right boundaries of the cell and the vertical lines.

[0174] Based on the number of rows and columns calculated earlier, a new Excel worksheet object is created for the above reconstructed table to obtain the structured recognition result.

[0175] As a further preferred embodiment, in this embodiment, all detected horizontal lines and vertical lines are traversed, and those lines that do not constitute the actual cell boundary are excluded.

[0176] Specifically, the types of lines that do not constitute the actual cell boundary are:

[0177] (1) Some false lines caused by image noise, poor scanning quality or errors of table line detection algorithm, such as paper boundary during scanning.

[0178] (2) Some lines similar to cell boundaries, such as dash "-" are likely to be identified as horizontal boundaries of cells.

[0179] (3) Partially missing table lines, which are incomplete lines caused by broken or blurred table lines.

[0180] Methods for excluding lines that do not constitute actual cell boundaries include:

[0181] (1) Length-based filtering: set a minimum length threshold to filter out lines with a length less than the threshold.

[0182] (2) Intersection-based filtering: only keep lines that intersect with other lines.

[0183] (3) Range-based filtering: only keep lines within the table boundary.

[0184] S8, the filled Excel table is subjected to general verification and hydrological yearbook data compliance verification, and after passing the verification, it is taken as the final recognition result.

[0185] General verification is applied to all types of tables in paper hydrological yearbook, mainly to eliminate errors in table format, such as multiple column merging. For multiple column merging, the first column in the merged column is retained and the other columns are deleted.

[0186] Multiple column merging mainly refers to single column data that becomes multiple columns after recognition. In simple terms, data that can be represented in one column is expressed in the form of merged cells in the recognition result.

[0187] The specific verification method is:

[0188] Check if there are empty rows or columns in the table. If a column is detected as a merged cell format and the value is empty, delete the column and merge the subsequent columns with the previous column.

[0189] Special verification refers to applying separate verification rules for each type of table to verify whether the character recognition is correct and whether it conforms to the logic.

[0190] As a preferred embodiment, in this embodiment, the hydrological yearbook data compliance verification of the filled Excel table includes multiple or all of the following verifications:

[0191] For the daily precipitation table, in the numerical area, it is verified whether all cells are legal characters, in the statistical value area, it is verified whether the precipitation is consistent with the corresponding month, whether the number of precipitation days is correct, whether the maximum daily amount is consistent with the numerical area, and whether the sum of the annual statistics and the numerical area is consistent;

[0192] For the maximum precipitation table of each period, it is verified whether all characters are legal characters, whether the station name exists, and whether the long-period precipitation is greater than or equal to the short-period maximum precipitation.

[0193] If all verifications pass, the final version of the table recognition result is generated; if there are problems in part of the verification, the verification problems are added to the verification result document for subsequent manual review.

[0194] An additional embodiment of the present application also provides a computer device comprising a memory and a processor, the memory storing computer instructions, and the processor executing the computer instructions to perform the paper hydrological yearbook table structured identification method disclosed in the foregoing embodiments.

[0195] An additional embodiment of the present application also provides a computer readable storage medium storing computer instructions, which, when executed by a computer, perform the paper hydrological yearbook table structured identification method disclosed in the foregoing embodiments.

[0196] It should be noted that the method of the embodiments of the present application can be executed by a single device, such as a computer or a server. The method of the present embodiment can also be applied to a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present application, and the multiple devices can interact with each other to complete the method.

[0197] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order described above and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0198] Embodiments of the present application are intended to cover all such alternatives, modifications, and variations as fall within the broad scope of the appended claims. Accordingly, any one of the above-described embodiments of the present application can be further modified through various illustrative modifications and changes in the specifics thereof without departing from the scope of the present application. Accordingly, such modifications and changes are intended to fall within the scope of the appended claims.

Claims

1. A paper hydrological yearbook table form structure recognition method, characterized in that, The method comprises the following steps: obtaining a scanned file of a paper hydrological yearbook and converting it into a picture file; obtaining a table image and attribute information of each converted picture file, wherein the attribute information at least comprises title text information and page number information of the table image; performing image preprocessing on the obtained table image, and performing correction and super-resolution conversion; performing straight line detection on the converted table image, and generating a table cell based on the straight line detection result, comprising: adding a table frame line to a position where a table line is missing on the table image, specifically comprising: traversing all rows of the table image starting from the first row, and calculating the distance between non-blank pixels in each row; for a certain row, if there are pixel pairs with a distance within a preset distance range and not adjacent, and the number of such pixel pairs exceeds a preset number threshold, it is determined that the row is a text row, otherwise it is a non-text row; applying morphological closing operation to remove black dots or black lines of the non-text row; calculating the connected regions by using the connected region operator of OpenCV, obtaining the height information of the connected regions, and sorting the connected regions in descending order of height; for the text row, the connected regions are calculated by using the connected region operator of OpenCV, the height of each connected region is obtained, the maximum row height of the front preset proportion and the minimum row height of the rear preset proportion are removed, and the average height of the remaining connected regions is calculated as the text row height; compare the connected regions of the non-text row with the text row height one by one, if the height is less than the text row height, draw a straight line at half the height of the connected region of the non-text row, otherwise do not draw a line, and obtain a table image with added table frame lines; performing preprocessing on the original table image by using a morphological dilation algorithm, so that the dotted line and the dotted line in the table image become continuous to form complete lines, and performing straight line detection on the preprocessed image; adopting a straight line merging strategy based on distance and angle to merge the detected straight lines to obtain table lines; based on the detected table lines, calculating the intersection points of all table lines, and generating cells based on the intersection points; performing merged cell and multi-layer nested structure recognition on the generated table cells, wherein the merged cell recognition comprises: for horizontal table lines, checking whether multiple intersection points have the same x coordinate in the vertical direction; for vertical table lines, checking whether multiple intersection points have the same y coordinate in the horizontal direction; if multiple intersection points meeting the condition are detected, it is determined that there is a merged cell across rows or columns; for the merged cell across rows, finding the minimum and maximum y coordinates of the multiple intersection points meeting the condition, and taking them as the upper and lower boundaries of the cell; for the merged cell across columns, finding the minimum and maximum x coordinates of the multiple intersection points meeting the condition, and taking them as the left and right boundaries of the cell; recognizing the text in the table image based on an OCR technology, obtaining the bounding box of each text region, and matching the obtained bounding box of the text region with its nearest neighbor cell; converting the coordinates of the cell after text matching into row and column serial numbers, mapping to an Excel table form, and filling the text recognition result into the Excel table cell. The filled Excel table is subjected to general verification and hydrological yearbook data compliance verification, and after the verification passes, it is taken as the final identification result, and the hydrological yearbook data compliance verification includes multiple or all of the following verifications: For the daily precipitation table, verify whether all cells are legal characters in the numerical area, verify whether the precipitation is consistent with the corresponding month, verify whether the precipitation days are correct, verify whether the maximum daily amount is consistent with the numerical area, and verify whether the annual statistics and the sum of the numerical area are consistent; For each period maximum precipitation table, verify whether all characters are legal characters, verify whether the station name exists, and verify whether the long period precipitation is greater than or equal to the short period maximum precipitation.

2. The paper hydrological yearbook table structure recognition method of claim 1, wherein, The scanning file of the paper hydrological yearbook is converted into a picture file, which includes: Obtaining a PDF scanning file of a paper hydrological yearbook through on-site scanning or reading; Analyzing the PDF scanning file to obtain size information and determining the file format based on the size information; Based on the file format and the preset picture parameters, the PDF scanning file is rendered into a single picture file.

3. The paper hydrologic yearbook table structure recognition method of claim 1, wherein, The table image and its attribute information in each converted picture file are obtained, which includes: Detecting the position of the table in the picture based on a pre-trained table region detection model to obtain a table image; Based on a pre-trained title detection model, a title picture is cropped from the picture, and the text in the title picture is recognized by a text recognition algorithm to obtain title text information; Based on a pre-trained page number detection model, a page number picture is cropped from the picture, and the text in the page number picture is recognized by a text recognition algorithm to obtain page number information; The title text information and the page number information are associated with the table image as attribute information of the table.

4. The paper hydrologic almanac table structure recognition method of claim 1, wherein, The table image is subjected to image preprocessing, correction and super-resolution conversion, which includes: Image preprocessing of the table image, including image graying, image filtering and image binarization; Angle correction and distortion correction of the preprocessed table image; Based on a pre-trained convolutional neural network, the super-resolution of the corrected table image is realized.

5. The paper hydrologic almanac table structure recognition method of claim 1, wherein, The straight lines detected are merged to obtain table lines by using a distance and angle based straight line merging strategy, which includes: Iterate through all detected straight lines, and for each pair of straight lines, calculate the minimum vertical distance and angle difference between them; If the distance between the two straight lines is less than a preset threshold and the angle difference is also less than a preset threshold, it is judged that the two straight lines are actually one, and they are merged into a new straight line; The merged straight line is placed at the center position of the original two straight lines, and the original length and direction characteristics are maintained.

6. The paper hydrologic almanac table structure recognition method of claim 1, wherein, The generated table cells are subjected to multi-layer nested structure identification, which includes: First, identify the outermost closed area as the starting point of the nested structure, and record the coordinates of the four corner points of the outermost closed area; Further detect the internal table lines and intersection points in the identified outermost closed area, and recursively call the closed area detection algorithm to analyze the internal nested structure layer by layer; When it is detected that a certain closed area no longer contains any table lines or intersection points, the recursion terminates, and the closed area is the bottom layer of a single cell.

7. The paper hydrological yearbook table structuring recognition method of claim 1, wherein, The converting the coordinates of the matched cells into row and column numbers and mapping to the Excel table form comprises: Defining a reference frame, selecting the first valid cell in the upper left corner of the table as the origin, and then determining the row and column of other cells according to the relative position between each horizontal line and vertical line; Counting the number of all detected horizontal lines and vertical lines respectively, and determining the total number of rows and columns of the entire table based on this; For each cell, calculate which row and column it belongs to according to the position of its four corner points relative to the reference frame; Based on the calculated number of rows and columns, create a new Excel worksheet object to reconstruct the table.

8. The paper hydrological yearbook table structure recognition method of claim 7, wherein, The traversing all detected horizontal lines and vertical lines further comprises filtering the detected straight lines based on the following rules: Length-based filtering: set a minimum length threshold, and filter out lines with a length less than the threshold; Intersection-based filtering: only keep lines that intersect with other lines; Range-based filtering: only keep lines within the table boundary.

9. A computer device, comprising: A memory and a processor, the memory stores computer instructions, and the processor executes the computer instructions to perform the paper hydrological yearbook table structure identification method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions, and the computer instructions are run by the computer to perform the paper hydrological yearbook table structure identification method according to any one of claims 1-8.

Images