Paper hydrological yearbook table structured recognition method and device and storage medium

Abstract

The invention provides a structured recognition method for a paper hydrological yearbook table. The structured recognition method comprises the following steps: acquiring a scanning file of a paper hydrological yearbook and converting the scanning file into a picture file; obtaining a table image and attribute information thereof in each converted picture file; carrying out image preprocessing, correction and super-resolution conversion on the obtained table image; performing straight line detection on the converted table image to generate table cells; the generated table cells are subjected to cell merging and multi-layer nested structure recognition; identifying characters in the table image, acquiring a bounding box of each character area, and matching the acquired bounding boxes of the character areas with nearest cells; the coordinates of the cells after character matching are converted into row and column serial numbers, the row and column serial numbers are mapped to an Excel table form, and the Excel table cells are filled with character recognition results; and performing general verification and hydrological yearbook data compliance verification on the filled Excel table, and taking the result as a final identification result after the verification is passed.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and table recognition, and in particular to a method, device and storage medium for structured recognition of a paper hydrological yearbook table. Background Art

[0002] As a valuable carrier of hydrological test data, the hydrological yearbook carries rich information on the long-term evolution of nature and the profound imprint of human activities, reflecting the changes in the natural environment and the impact of human activities. However, traditional paper hydrological yearbooks have many inconveniences in preservation and use, such as easy damage, difficult retrieval and analysis. Digitizing paper hydrological yearbook tables can greatly improve the efficiency of data retrieval and use. Traditional paper materials need to be manually flipped through, which is time-consuming and error-prone, while digitized data can be quickly retrieved by computer, providing instant information support for researchers and decision makers. In addition, digitized hydrological data facilitates large-scale data analysis and model construction, which helps to deeply explore the scientific laws behind the data and predict future hydrological trends.

[0003] However, compared with ordinary tables, the automatic recognition of hydrological yearbook tables faces multiple difficulties: 1) The table size of hydrological yearbooks is usually large, reaching more than 1000×2000 pixels, and the aspect ratio is not fixed, which brings challenges to image processing and recognition.

[0004] 2) There are many cells in the hydrological yearbook table images, and the text information is dense, which requires the recognition algorithm to have a high degree of accuracy and precision. However, there is often a lack of sufficient table lines in the table images, which makes it particularly difficult to reconstruct the table structure. In addition, the existence of a large number of blank cells and blank columns, and the lack of table lines as a reference, further increases the difficulty of table reconstruction and text positioning. To make it more complicated, the hydrological yearbook table image may contain two or more tables, and each table area has a title, which destroys the overall structure of the table and increases the complexity of recognition. At the same time, the structure of the hydrological yearbook table itself may also be very complex, including merging cells, dividing sub-tables, and the presence of sub-titles in the table content, all of which require the recognition algorithm to be able to flexibly cope with various complex situations.

[0005] 3) The image quality of hydrological yearbook tables is also an issue that cannot be ignored. The lines of the tables may be curved, the angles of the tables may be incorrect, and the text may be blurred, all of which bring additional challenges to accurate table recognition. Summary of the invention

[0006] In view of the difficulty in automatically identifying table images in paper hydrological yearbooks, the present invention proposes a method for structured recognition of paper hydrological yearbook tables. It comprehensively utilizes image processing technology, target detection technology, text recognition technology, table structured recognition technology, etc., and performs structured recognition on complex paper hydrological yearbook table images. It can process various table types in paper hydrological yearbooks and overcome the problem of low table image quality caused by factors such as poor quality of paper hydrological yearbooks and scanning.

[0007] To achieve the above object, the present invention adopts the following technical solutions: The first aspect of the present invention discloses a method for identifying a structured table in a paper hydrological yearbook, comprising the following steps: Obtain scanned files of paper hydrological yearbooks and convert them into image files; Acquire a table image and its attribute information in each converted image file, wherein the attribute information at least includes title text information and page number information of the table image; Perform image preprocessing on the acquired table image, and perform correction and super-resolution conversion; Perform straight line detection on the converted table image, and generate table cells based on the straight line detection results; Identify the merged cells and multi-layer nested structures of the generated table cells; Recognize text in the table image based on OCR technology, obtain the bounding box of each text area, and match the obtained bounding box of the text area with its nearest neighbor cell; Convert the coordinates of the cells after text matching into row and column numbers, map them into Excel table format, and fill the text recognition results into the Excel table cells; The filled Excel table is subjected to general verification and hydrological yearbook data compliance verification, and is used as the final identification result after passing the verification.

[0008] Furthermore, the step of converting the scanned file of the paper hydrological yearbook into a picture file includes: Obtain PDF scanned files of the paper hydrological yearbook by on-site scanning or reading; Parsing the PDF scanned file to obtain size information, and determining the file format based on the size information; Renders scanned PDF files into a single image file based on the file layout and preset image parameters.

[0009] Furthermore, the step of obtaining the table image and its attribute information in each converted picture file includes: Detect the position of the table in the image based on the pre-trained table area detection model to obtain a table image; Based on the pre-trained title detection model, the title image is cropped from the image, and the text in the title image is recognized by the text recognition algorithm to obtain the title text information; Based on the pre-trained page number detection model, the page number image is cropped from the image, and the text in the page number image is recognized by the text recognition algorithm to obtain the page number information; The title text information and page number information are associated with the table image as attribute information of the table.

[0010] Furthermore, the image preprocessing, correction and super-resolution conversion of the acquired table image includes: Performing image preprocessing on the table image, including image graying, image filtering and image binarization; Performing angle correction and distortion correction on the preprocessed table image; Super-resolution of rectified table images is achieved based on a pre-trained convolutional neural network.

[0011] Furthermore, the performing straight line detection on the converted table image and generating table cells based on the straight line detection result includes: Adding a table frame line at a position where the table line is missing on the table image; The morphological dilation algorithm is used to preprocess the original table image, so that the dotted lines and dot-dashed lines in the table image become continuous and form complete lines, and straight line detection is performed in the preprocessed image; The detected straight lines are merged using a straight line merging strategy based on distance and angle to obtain a table line; Based on the detected table lines, calculate the intersection points of all table lines and generate cells based on the intersection points.

[0012] Furthermore, the adding of table border lines at positions where table border lines are missing on the table image includes: Detect non-text rows in a table image, traverse all rows of the table image starting from the first row, and calculate the distance between non-blank pixels in each row; for a row, if there are non-adjacent pixel pairs whose distance is within a preset distance range, and the number of such pixel pairs exceeds a preset threshold, then the row is determined to be a text row, otherwise it is a non-text row; Apply morphological closing operation to remove black dots or black line segments that are not text lines; calculate the connected area through the connected area operator of OpenCV, obtain the height information of the connected area, and sort the connected areas from large to small according to their height; For text lines, the connected area is calculated using the connected area operator of OpenCV, and the height of each connected area is obtained. The maximum line height of the previous preset ratio and the minimum line height of the latter preset ratio are removed, and the average height of the remaining connected areas is calculated as the text line height; Compare the connected areas of non-text rows with the text row height one by one. If the height is less than the text row height, draw a straight line at half the height of the connected areas of non-text rows. Otherwise, no line is drawn, and a table image with a table frame line added is obtained.

[0013] Furthermore, the method of merging the detected straight lines to obtain the table lines by using the straight line merging strategy based on distance and angle includes: Traverse all detected lines, and for each pair of lines, calculate the minimum vertical distance and angle difference between them; If the distance between two straight lines is less than a preset threshold and the angle difference is also less than a preset threshold, it is determined that the two straight lines are actually one, and they are merged into a new straight line; Place the merged line at the center of the two original lines, while maintaining the original length and direction characteristics.

[0014] Furthermore, the merged cell recognition of the generated table cells includes: For horizontal table lines, check whether there are multiple intersections with the same x coordinate in the vertical direction; for vertical table lines, check whether there are multiple intersections with the same y coordinate in the horizontal direction; If multiple intersections that meet the conditions are detected, it is determined that there are merged cells that span rows or columns; For merged cells across rows, find the minimum and maximum y coordinates of multiple intersections that meet the conditions and use them as the upper and lower boundaries of the cell; for merged cells across columns, find the minimum and maximum x coordinates of multiple intersections that meet the conditions and use them as the left and right boundaries of the cell.

[0015] Furthermore, the identification of the multi-layer nested structure of the generated table cells includes: First, identify the outermost closed area as the starting point of the nested structure, and record the coordinates of the four corner points of the outermost closed area; In the outermost closed area identified, the internal table lines and intersections are further detected, and the internal nested structure is analyzed layer by layer by recursively calling the closed area detection algorithm; The recursion terminates when it is detected that a closed area no longer contains any table lines or intersections. The closed area is the single cell at the bottom layer.

[0016] Furthermore, the conversion of the coordinates of the cells after text matching into row and column numbers and mapping them into an Excel table format includes: Define the reference frame, select the first valid cell in the upper left corner of the table as the origin, and then determine the rows and columns where other cells are located based on the relative positions between various horizontal and vertical lines; Traverse all detected horizontal and vertical lines, count the number of them and use this as a basis to determine how many rows and columns the entire table has; For each cell, calculate the row and column it belongs to based on the positions of its four corner points relative to the reference frame; Create a new Excel worksheet object to rebuild the table based on the calculated number of rows and columns.

[0017] Furthermore, the traversing of all detected horizontal lines and vertical lines further includes filtering the detected straight lines based on the following rules: Length-based filtering: set a minimum length threshold to filter out lines whose length is less than the threshold; Intersection-based filtering: only keep lines that intersect with other lines; Range-based filtering: only keep lines that are within the table boundaries.

[0018] Furthermore, the compliance check of the hydrological yearbook data on the filled Excel table includes multiple or all of the following checks: For the daily precipitation table, verify whether all cells in the value area are legal characters, verify whether the precipitation is consistent with the corresponding lunar phase in the statistical value area, verify whether the number of precipitation days is correct, verify whether the maximum daily amount is consistent with the value area, and verify whether the total of the annual statistics is consistent with the total value area; For the maximum precipitation table for each time period, verify whether all characters are legal characters, verify whether the station name exists, and verify whether the long-term precipitation is greater than or equal to the short-term maximum precipitation.

[0019] The second aspect of the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores computer instructions, and the processor executes the method for structured recognition of paper hydrological yearbook tables as described in the first aspect above by executing the computer instructions.

[0020] The third aspect of the present invention further discloses a computer-readable storage medium storing computer instructions, which, when executed by a computer, execute the method for identifying the structure of a paper hydrological yearbook table as described in the first aspect.

[0021] The beneficial technical effects of the present invention are as follows: 1) The paper hydrological yearbook table structured recognition method of the present invention can realize the full-process automated processing of the hydrological yearbook table from paper scanned PDF files to structured recognition, and provides an effective technical means for the digital processing of paper hydrological yearbook tables.

[0022] 2) The paper hydrological yearbook table structured recognition method of the present invention has a wide range of applications and can process various tables in all types of paper hydrological yearbooks.

[0023] 3) The paper hydrological yearbook table structured recognition method of the present invention can achieve good recognition effect even for table images of poor quality caused by factors such as poor quality of paper yearbooks or scanning, through table image preprocessing, angle and distortion correction and super-resolution conversion.

[0024] 4) The paper hydrological yearbook table structured recognition method of the present invention can automatically determine whether there is a missing table frame line, and can automatically add table lines at the position where the table frame line is missing to form a framed table, thereby reducing the recognition difficulty of a frameless table, and can also achieve high accuracy recognition of hydrological yearbook tables with missing table lines.

[0025] 5) The paper hydrological yearbook table structured recognition method of the present invention is targeted at the merged cells and complex nested structures of cells that are commonly found in hydrological yearbook tables. It can effectively identify merged cells through collinearity detection and gradually restore the nested cell structure through recursive inner cells, thereby accurately reconstructing the table structure.

[0026] 6) The paper hydrological yearbook table structured recognition method of the present invention can perform a comprehensive verification of the automatic recognition results through universal verification and hydrological yearbook data compliance verification, which can not only repair the unreasonable table structure, but also effectively ensure the accuracy of the recognition results, and greatly reduce the time and labor cost of manual cell-by-cell verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The figure is a flow chart of an embodiment of a method for identifying structured tables in a paper hydrological yearbook according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0029] See also Figure 1 The embodiment of the present invention discloses a method for identifying a structured table in a paper hydrological yearbook, comprising the following steps: S1. Obtain the scanned files of the paper hydrological yearbook and convert them into image files.

[0030] As a preferred implementation scheme, in this embodiment, this step is implemented using the following process: S11. Obtain the PDF scanned file of the paper hydrological yearbook by on-site scanning or reading.

[0031] Specifically, the method of obtaining the paper hydrological yearbook PDF scan file may be on-site scanning, or reading a stored scanned PDF scan file, which is not specifically limited in the present invention.

[0032] S12, parsing the PDF scanned file, obtaining size information, and determining the file format based on the size information.

[0033] Specifically, the PDF file needs to be parsed to obtain the width and height of the PDF file.

[0034] The PDF files scanned from the paper hydrological yearbook usually have two formats, A3 and A4. The A3 format contains two pages of paper, while the A4 format contains only a single page. If the width of the PDF file is greater than the height, it means that the PDF file is in A3 format. In the subsequent rendering step, one page in the PDF file needs to be rendered into two images, each corresponding to a single page of paper; if the height of the PDF is greater than the width, it means that the PDF file is in A4 format. In the subsequent rendering step, one page in the PDF file is rendered into one image, which is also a single page of the paper book.

[0035] S13. Render the PDF scanned file into a single image file based on the file layout and preset image parameters.

[0036] Specifically, if a hydrological paper yearbook has 100 pages, that is, 50 sheets of paper, then the final rendering result is 100 images. In an A4-format PDF file, it is a 100-page PDF file, and the 100-page PDF file is rendered into 100 images; in an A3-format PDF file, it is a 50-page PDF file, and one page in the A3-format PDF file needs to be rendered into two images, that is, the 50-page PDF file is rendered into 100 images.

[0037] When rendering each page of a PDF file into an image, you need to set the following parameters: ① Picture format: Common formats include jpg and png; ② Image size: such as 2480 pixels wide and 3509 pixels high; ③ Image DPI resolution: conventional DPI is 100 / 200 / 300; ④ Color space: There are conventional RGB and CMYK spaces. Set the RGB space here; ⑤ Number of image channels: The conventional number is three channels.

[0038] Among them, the two parameters of image size and DPI resolution are set either one or the other. If the image size is set, there is no need to set the DPI resolution. If the DPI is set, there is no need to set the image size.

[0039] S2. Obtain the table image and its attribute information in each converted image file, wherein the attribute information at least includes the title text information and page number information of the table image.

[0040] Generally speaking, a page in a hydrological yearbook usually has one or more tables, each table has a title, and each page has a page number.

[0041] As a preferred implementation scheme, in this embodiment, this step is implemented using the following process: S21. Detect the position of the table in the image based on the pre-trained table area detection model to obtain a table image.

[0042] In an illustrative example, the table area detection model used in the present invention is trained in the following manner: 1) Construct a dataset for table area detection, which is used to train the table area detection model. The dataset consists of rendered images and labels. The number is unlimited, the more the better, and usually more than 100 images. It is usually considered that the upper left corner of the image is the starting origin (0,0). x and y represent the position of any pixel in the image, respectively. x is the width direction and y is the height direction. The table area can be represented by a rectangle in the image. There are many ways to represent the rectangle. Here are two examples: ① The coordinates of the upper left corner and the lower right corner of the rectangle, which can form a rectangle; ② The coordinates, width, and height of the upper left corner of the rectangle. The label is used to record the location information of each table area. If there is a table in an image, and the rectangle is represented by the above method ①, then the label corresponding to this image records the coordinates of the upper left corner (x1, y1) and the coordinates of the lower right corner (x2, y2). The label can be in many forms and can be saved in txt format, xml format, or json format.

[0043] 2) Train the table area detection model. The model can be built based on a neural network model such as Faster R-CNN, YOLO, or DETR. Set the parameters required by the model. For example: ① Epochs: how many times the model trains the dataset; ② Batch size: How many images are input at a time during training; ③ Learning rate: determines the step size of the model’s parameter update at each iteration; ④ Loss function: measures the difference between the model’s predicted value and the true value; ⑤ Optimizer: How to adjust its internal parameters to reduce errors based on the results of the loss function; ⑥ Weight initialization: initialization method of model parameters.

[0044] After setting the parameters, train the model according to the set parameters. Before training, you must first select an initialization method to initialize the weights. Training is done in small batches, with Batchsize images input at a time. An iteration is completed only after all images in the data set are traversed. During training, input image X and label Y_true to the model. The model determines the position Y_pred of the table in image X for the input image X. The loss function calculates the difference between Y_true and Y_pred. The optimizer uses the learning power and the calculated difference value to update the model's weights.

[0045] After the training is completed, the trained table area detection model can be used to detect the position of the table in each image to be detected. If the label in the data set is expressed in method ①, the result of the model detection is also expressed in the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2). By cutting the table area from the image page through (x1, y1) and (x2, y2), a table image can be obtained.

[0046] S22. Based on the pre-trained title detection model, a title image is cropped from the image, and the text in the title image is recognized by a text recognition algorithm to obtain title text information.

[0047] Specifically, the training of the title detection model can refer to the training method of the above table area detection model. The difference is that the label in the data set represents the position information of the title in the picture.

[0048] After the training is completed, the trained title detection model can be used to crop the title image from the image to be detected, and then the OCR recognition algorithm can be used to recognize the text in the title image to obtain the text information of the title.

[0049] S23. Based on the pre-trained page number detection model, a page number image is cropped from the image, and the text in the page number image is recognized by a text recognition algorithm to obtain page number information.

[0050] Specifically, the training of the page number detection model can also refer to the training method of the above-mentioned table area detection model, which will not be described in detail here.

[0051] After the training is completed, the trained page number detection model can be used to crop the page number image from the image to be detected, and then the OCR recognition algorithm can be used to recognize the text in the page number image to obtain the page number information.

[0052] S24. Associating the title text information and page number information with the corresponding table image as attribute information of the table.

[0053] It should be noted that there is no sequential order among the above steps S21, S22, and S23. During implementation, the detection order can be adjusted according to actual conditions, which does not affect the implementation of the solution of the present invention.

[0054] S3. Preprocess the acquired table image and perform correction and super-resolution conversion.

[0055] As a preferred implementation scheme, in this embodiment, this step is implemented using the following process: S31, perform image preprocessing on the table image, including image grayscale, image filtering and image binarization. All table images need to be preprocessed, and the main purpose is to enhance the features in the table image.

[0056] Specifically, image grayscale is the process of converting a color image into a grayscale image. Each pixel in a grayscale image has only a brightness value and no color information. The RGB value of a color image is converted into a grayscale image using the average method, and different weights are assigned to the three RGB components, and then the weighted sum is calculated as the grayscale value. For example, commonly used weights are 0.299 (red), 0.587 (green), and 0.114 (blue).

[0057] Image filtering is the process of suppressing the noise of the target image while preserving the image details as much as possible to improve the image quality. Select a suitable filter, such as a mean filter, Gaussian filter, median filter, etc., and set the corresponding parameters, such as the filter size, standard deviation, etc. Try different filters and corresponding parameters many times, select the best filter and parameters, and get the image after image filtering.

[0058] Image binarization is to convert an image into an image containing only two pixel values. Adaptive threshold method is used to dynamically select the threshold according to the local features of the image to obtain the binarized image.

[0059] S32, performing angle correction and distortion correction on the preprocessed table image.

[0060] Due to improper storage of paper hydrological yearbooks or distortion during scanning, the table image may be rotated to a certain extent, or the table may be distorted. In view of this situation, the present invention solves the problem by first correcting the angle of the table and then correcting the distortion.

[0061] Specifically, the process of angle correction is: use edge detection operator or Hough transform to find the main lines in the image, calculate the angle that the image needs to be rotated according to the direction of the main lines, and then use image rotation transformation to correct the image angle.

[0062] The distortion correction process is as follows: use the edge detection operator to detect continuous line segments, and calculate the coordinates of the cross intersections of each continuous line segment and other line segments. The coordinates of each intersection on the same continuous line segment on the horizontal / vertical axis are used to calculate the coordinates of each intersection in the non-distorted state. The image projection transformation algorithm is applied to map the distorted image to a non-distorted image.

[0063] S33. Super-resolution of the corrected table image is achieved based on a pre-trained convolutional neural network.

[0064] In view of the compact characters, dense text information, unclear characters, etc. of the paper hydrological yearbook, the present invention applies a deep learning super-resolution algorithm to repair it. Specifically, the table image size is enlarged to twice the size of the original image, and the mapping relationship from low-resolution image to high-resolution image is learned through a convolutional neural network. The features of the learned low-resolution image are converted into features of the high-resolution image using deep convolution and pixel reconstruction technology, thereby achieving super-resolution of the image.

[0065] S4. Perform straight line detection on the converted table image, and generate table cells based on the straight line detection results.

[0066] As a preferred implementation scheme, in this embodiment, this step is implemented using the following process: S41. Add table border lines at the positions where table lines are missing on the table image.

[0067] Aiming at the problem that a large number of table lines are missing in the paper hydrological yearbook, the present invention solves the problem by automatically adding table frame lines at the positions where the table lines are missing on the table image. The specific process is as follows: Detect non-text rows in table images. Traverse all rows of the table image starting from the first row and calculate the distance between non-blank pixels in each row. For a row, if there are non-adjacent pixel pairs whose distance is within the preset distance range (for example, the pixel distance is between 3 and 20), and the number of such pixel pairs exceeds the preset number threshold (for example, more than 3), then the row is determined to be a text row, otherwise it is a non-text row.

[0068] Apply morphological closing operation to remove black dots or black line segments that are not text lines. Calculate the connected area through OpenCV's connected area operator to obtain the height information of the connected area. Sort the connected areas from large to small according to their height.

[0069] For text lines, the connected areas are calculated using the connected area operator of OpenCV to obtain the height of each connected area. The maximum line height of the first 5% and the minimum line height of the last 5% are removed, and the average height of the remaining connected areas is calculated as the text line height.

[0070] The connected areas of non-text rows are compared with the text row height one by one. If the height is less than the text row height, a straight line is drawn at half the height of the connected area of ​​the non-text row. Otherwise, no line is drawn to obtain a table image with lines.

[0071] S42. Preprocess the original table image using a morphological dilation algorithm, and perform straight line detection in the preprocessed image.

[0072] In order to improve the continuity and detectability of the table lines, the morphological dilation algorithm is used in this embodiment to pre-process the original table image. The dilation operation is a basic morphological operation of binary images. It connects the disconnected parts by expanding the foreground objects (i.e., white pixels), so that the dotted lines and dot-dashed lines become continuous and form complete lines. The specific implementation process is as follows: (1) Convert the table image to be processed into a binary image, in which the table lines are represented as white and the background is black; (2) Use an appropriate structural element to perform the dilation operation to ensure that the dashed lines and dot-dashed lines can be effectively connected, but do not over-expand to reduce the distance between adjacent lines; (3) Perform necessary cleaning on the expanded image, such as removing isolated small dots or noise, to ensure the quality of subsequent processing.

[0073] After preprocessing, the next step is to detect the straight lines in the image. In this embodiment, Hough Transform is used for straight line detection. This is a classic feature extraction technology that is particularly suitable for detecting regular shapes such as straight lines. Hough Transform can map points in the image space to curves in the parameter space, thereby realizing straight line detection. The specific implementation scheme is: (1) Apply the Hough transform algorithm to analyze the expanded image and find possible straight lines; (2) Screen out significant straight line segments based on the set threshold. These straight line segments represent the actual boundaries of the table. (3) Calculate the position and direction parameters of each straight line for further processing.

[0074] S43, using a straight line merging strategy based on distance and angle to merge the detected straight lines to obtain a table line.

[0075] Considering that there may be multiple very close parallel lines in the actual table, this may be caused by factors such as printing quality and scanning errors. In order to simplify the table structure and reduce redundant information, the present invention uses a straight line merging strategy based on distance and angle: (1) Traverse all detected straight lines and, for each pair of straight lines, calculate the minimum vertical distance and angle difference between them; (2) If the distance between two straight lines is less than a preset threshold and the angle difference is also less than a preset threshold, it is determined that the two straight lines are actually one and they are merged into a new straight line; (3) The merged straight line should be located at the center of the original two straight lines and maintain the original length and direction characteristics.

[0076] S44. Based on the detected table lines, calculate the intersection points of all table lines, and generate cells based on the intersection points.

[0077] Based on the table line detection, the intersection points of all table lines are calculated, and the intersection points can be used to form cells. In this embodiment, the following method is used: First, all detected lines need to be parameterized. Usually, the slope-intercept form (y = mx + b) or the general form (Ax + By + C = 0) can be used to represent lines. If two lines are not parallel, they must intersect at one point.

[0078] It should be noted that not all calculated intersections are valid, because some intersections may be caused by image noise or false detection. Therefore, it is necessary to set some rules to filter out these invalid intersections according to the actual situation, for example: only keep the intersections within the image boundary; exclude new intersections that are very close to other known intersections to avoid repeated counting.

[0079] Once all valid intersections are obtained, an adjacency graph can be constructed to represent the connections between adjacent intersections. In this graph, each node represents an intersection, and an edge represents two intersections connected by the same straight line. By traversing the adjacency graph, all closed areas are found, that is, a set of spaces completely surrounded by straight line segments. These closed areas are the cells in the table. For each possible cell, the location information of its four corner points is recorded for subsequent processing.

[0080] S5. Recognize the merged cells and multi-layer nested structures of the generated table cells.

[0081] For some complex table layouts, especially the hydrological yearbook table targeted by the present invention, there will be many cases such as merged cells or multi-layer nesting, and a simple closed area search may not be enough to correctly split all cells. In this case, the following strategy can be adopted: (1) For merged cells that span multiple rows or columns, check whether there are multiple intersections on a single line and adjust the cell boundaries accordingly. When multiple intersections are detected on a single line, it can be inferred that there are merged cells. For example, if a cell spans multiple rows, there will be multiple intersections on the same straight line in the vertical direction. At this time, the range of the merged cells can be correctly identified by adjusting the cell boundaries.

[0082] As a preferred implementation scheme, in this embodiment, identifying merged cells of the generated table cells includes: For horizontal table lines, check whether there are multiple intersection points that are collinear in the vertical direction, that is, have the same x coordinate; for vertical table lines, check whether there are multiple intersection points that are collinear in the horizontal direction, that is, have the same y coordinate; If multiple intersections that meet the collinearity condition are detected, it is determined that there are merged cells that span rows or columns; For merged cells across rows, find the minimum and maximum y coordinates of multiple intersections that meet the collinearity condition and use them as the upper and lower boundaries of the cell; for merged cells across columns, find the minimum and maximum x coordinates of multiple intersections that meet the collinearity condition and use them as the left and right boundaries of the cell.

[0083] (2) For multi-layer nested table structures, a recursive method can be used for analysis. First, identify the outermost closed area, and then further search for the inner closed area within the area. Recursively analyze each layer of closed areas until the single cell at the bottom layer is identified.

[0084] As a preferred implementation, in this embodiment, identifying the multi-layer nested structure of the generated table cells includes: First, identify the outermost closed area as the starting point of the nested structure, and record the coordinates of the four corner points of the outermost closed area; In the outermost closed area identified, the internal table lines and intersections are further detected, and the internal nested structure is analyzed layer by layer by recursively calling the closed area detection algorithm; The recursion terminates when it is detected that a closed area no longer contains any table lines or intersections. The closed area is the single cell at the bottom layer.

[0085] S6. Recognize the text in the table image based on OCR technology, obtain the bounding box of each text area, and match the obtained bounding box of the text area with its nearest neighbor cell.

[0086] As a preferred implementation scheme, in this embodiment, this step is implemented using the following process: Use OCR (Optical Character Recognition) technology to recognize the text in the table image and obtain the precise location of each text in the image (i.e., bounding box); The bounding box of each text area is matched with its nearest neighboring cell. In this embodiment, the geometric center distance minimization principle is adopted: for each text bounding box, the cell closest to its geometric center is found as the corresponding container.

[0087] S7. Convert the coordinates of the cells after text matching into row and column numbers, map them into Excel table format, and fill the text recognition results into the Excel table cells.

[0088] As a preferred implementation scheme, in this embodiment, this step is implemented using the following process: In order to easily convert cell coordinates into row and column numbers, a reference frame is first defined. Usually, the first valid cell in the upper left corner of the table is selected as the origin (0, 0), and then the rows and columns of other cells are determined based on the relative positions between the horizontal and vertical lines. Traverse all detected horizontal and vertical lines, count their number, and use this as a basis to determine how many rows and columns the entire table has. For each cell, calculate which row and column it belongs to based on the position of its four corner points relative to the reference frame. Specifically, the row number can be determined by comparing the positional relationship between the upper and lower boundaries of the cell and the horizontal line; similarly, the column number can be determined by the relationship between the left and right boundaries and the vertical line.

[0089] Based on the number of rows and columns calculated previously, a new Excel worksheet object is created from the above reconstructed table to obtain the structured recognition result.

[0090] As a further preferred real-time solution, in this embodiment, all detected horizontal and vertical lines are traversed, and those lines that do not constitute actual cell boundaries are excluded.

[0091] Specifically, the types of lines that do not constitute actual cell boundaries are: (1) Some false lines, such as paper boundaries, are generated due to image noise, poor scanning quality, or errors in the table line detection algorithm.

[0092] (2) When detecting lines that are similar to cell boundaries, for example, a dash “——” is likely to be recognized as a horizontal cell boundary.

[0093] (3) Partially missing table lines, which are incomplete lines formed due to the table lines being broken or blurred.

[0094] Ways to exclude lines that do not constitute actual cell boundaries include: (1) Length-based filtering: Set a minimum length threshold and filter out lines whose length is less than the threshold.

[0095] (2) Intersection-based filtering: only retain lines that intersect with other lines.

[0096] (3) Range-based filtering: only keep the lines within the table boundaries.

[0097] S8. Perform general verification and hydrological yearbook data compliance verification on the filled Excel table, and use the results after passing the verification as the final recognition result.

[0098] The general check is applied to all types of tables in the paper hydrological yearbook, mainly to eliminate errors in table format, such as the case of merging multiple columns. For the case of merging multiple columns, the first column of the merged column is retained and the other columns are deleted.

[0099] The situation of multi-column merging mainly refers to the situation where the data in a single column becomes multiple columns after recognition. In simple terms, the data that can be represented by one column is expressed in the form of merged cells in the recognition result.

[0100] The specific verification method is: Checks whether there are empty rows or columns in the table. If a column is detected to be in merged cell format and has an empty value, delete the column and connect the subsequent columns to the previous column.

[0101] Dedicated validation means applying separate validation rules to each form type to verify whether character recognition is correct and logical.

[0102] As a preferred implementation scheme, in this embodiment, the compliance check of the hydrological yearbook data on the filled Excel table includes multiple or all of the following checks: For the daily precipitation table, verify whether all cells in the value area are legal characters, verify whether the precipitation is consistent with the corresponding lunar phase in the statistical value area, verify whether the number of precipitation days is correct, verify whether the maximum daily amount is consistent with the value area, and verify whether the total of the annual statistics is consistent with the total value area; For the maximum precipitation table for each time period, verify whether all characters are legal characters, verify whether the station name exists, and verify whether the long-term precipitation is greater than or equal to the short-term maximum precipitation.

[0103] If all verifications are passed, the final version of the table recognition results will be generated; if there are problems with some verifications, the verification problems will be added to the verification result document for subsequent manual review.

[0104] Another embodiment of the present invention further provides a computer device, including a memory and a processor, wherein the memory stores computer instructions, and the processor executes the paper hydrological yearbook table structured recognition method disclosed in the above embodiment by executing the computer instructions.

[0105] Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a computer, the method for identifying the structure of a paper hydrological yearbook table disclosed in the above embodiment is executed.

[0106] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.

[0107] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0108] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying the structure of a paper hydrological yearbook table, characterized in that: The steps include: Obtain scanned files of paper hydrological yearbooks and convert them into image files; Acquire a table image and its attribute information in each converted image file, wherein the attribute information at least includes title text information and page number information of the table image; Perform image preprocessing on the acquired table image, and perform correction and super-resolution conversion; Perform straight line detection on the converted table image, and generate table cells based on the straight line detection results; Identify the merged cells and multi-layer nested structures of the generated table cells; Recognize text in the table image based on OCR technology, obtain the bounding box of each text area, and match the obtained bounding box of the text area with its nearest neighbor cell; Convert the coordinates of the cells after text matching into row and column numbers, map them into Excel table format, and fill the text recognition results into the Excel table cells; The filled Excel table is subjected to general verification and hydrological yearbook data compliance verification, and is used as the final identification result after passing the verification.

2. The method for identifying the structure of a paper hydrological yearbook table according to claim 1, characterized in that: The method of converting the scanned file of the paper hydrological yearbook into a picture file comprises: Obtain PDF scanned files of the paper hydrological yearbook by on-site scanning or reading; Parsing the PDF scanned file to obtain size information, and determining the file format based on the size information; Renders scanned PDF files into a single image file based on the file layout and preset image parameters.

3. The paper hydrological yearbook table structure recognition method according to claim 1, characterized in that: The step of obtaining the table image and its attribute information in each converted picture file includes: Detect the position of the table in the image based on the pre-trained table area detection model to obtain a table image; Based on the pre-trained title detection model, the title image is cropped from the image, and the text in the title image is recognized by the text recognition algorithm to obtain the title text information; Based on the pre-trained page number detection model, the page number image is cropped from the image, and the text in the page number image is recognized by the text recognition algorithm to obtain the page number information; The title text information and page number information are associated with the table image as attribute information of the table.

4. The method for identifying the structure of a paper hydrological yearbook table according to claim 1, characterized in that: The image preprocessing, correction and super-resolution conversion of the acquired table image include: Performing image preprocessing on the table image, including image graying, image filtering and image binarization; Performing angle correction and distortion correction on the preprocessed table image; Super-resolution of rectified table images is achieved based on a pre-trained convolutional neural network.

5. The method for identifying the structure of a paper hydrological yearbook table according to claim 1, characterized in that: The performing straight line detection on the converted table image and generating table cells based on the straight line detection result includes: Adding a table frame line at a position where the table line is missing on the table image; The morphological dilation algorithm is used to preprocess the original table image, so that the dotted lines and dot-dashed lines in the table image become continuous and form complete lines, and straight line detection is performed in the preprocessed image; The detected straight lines are merged to obtain the table lines by using the straight line merging strategy based on distance and angle; Based on the detected table lines, calculate the intersection points of all table lines and generate cells based on the intersection points.

6. The method for identifying the structure of a paper hydrological yearbook table according to claim 5, characterized in that: The step of adding a table frame line at a position where the table line is missing on the table image includes: Traverse all rows of the table image starting from the first row, and calculate the distance between non-blank pixels in each row; for a row, if there are non-adjacent pixel pairs whose distance is within a preset distance range, and the number of such pixel pairs exceeds a preset threshold, then the row is determined to be a text row, otherwise it is a non-text row; Apply morphological closing operation to remove black dots or black line segments that are not text lines; calculate the connected area through the connected area operator of OpenCV, obtain the height information of the connected area, and sort the connected areas from large to small according to their height; For text lines, the connected area is calculated using the connected area operator of OpenCV, and the height of each connected area is obtained. The maximum line height of the previous preset ratio and the minimum line height of the latter preset ratio are removed, and the average height of the remaining connected areas is calculated as the text line height; Compare the connected areas of non-text rows with the text row height one by one. If the height is less than the text row height, draw a straight line at half the height of the connected areas of non-text rows. Otherwise, no line is drawn, and a table image with a table frame line added is obtained.

7. The method for identifying the structure of a paper hydrological yearbook table according to claim 5, characterized in that: The method of merging the detected straight lines to obtain the table lines by using the straight line merging strategy based on distance and angle includes: Traverse all detected lines, and for each pair of lines, calculate the minimum vertical distance and angle difference between them; If the distance between two straight lines is less than a preset threshold and the angle difference is also less than a preset threshold, it is determined that the two straight lines are actually one, and they are merged into a new straight line; Place the merged line at the center of the two original lines, while maintaining the original length and direction characteristics.

8. The method for identifying the structure of a paper hydrological yearbook table according to claim 5, characterized in that: The step of identifying merged cells of the generated table cells includes: For horizontal table lines, check whether there are multiple intersections with the same x coordinate in the vertical direction; for vertical table lines, check whether there are multiple intersections with the same y coordinate in the horizontal direction; If multiple intersections that meet the conditions are detected, it is determined that there are merged cells that span rows or columns; For merged cells across rows, find the minimum and maximum y coordinates of multiple intersections that meet the conditions and use them as the upper and lower boundaries of the cell; for merged cells across columns, find the minimum and maximum x coordinates of multiple intersections that meet the conditions and use them as the left and right boundaries of the cell.

9. The method for identifying the structure of a paper hydrological yearbook table according to claim 5, characterized in that: The identification of the multi-layer nested structure of the generated table cells includes: First, identify the outermost closed area as the starting point of the nested structure, and record the coordinates of the four corner points of the outermost closed area; In the outermost closed area identified, the internal table lines and intersections are further detected, and the internal nested structure is analyzed layer by layer by recursively calling the closed area detection algorithm; The recursion terminates when it is detected that a closed area no longer contains any table lines or intersections. The closed area is the single cell at the bottom layer.

10. The method for identifying the structure of a paper hydrological yearbook table according to claim 1, characterized in that: The step of converting the coordinates of the cells after the text matching into row and column numbers and mapping them into an Excel table format includes: Define the reference frame, select the first valid cell in the upper left corner of the table as the origin, and then determine the rows and columns where other cells are located based on the relative positions between various horizontal and vertical lines; Traverse all detected horizontal and vertical lines, count the number of them and use this as a basis to determine how many rows and columns the entire table has; For each cell, calculate the row and column it belongs to based on the positions of its four corner points relative to the reference frame; Create a new Excel worksheet object to rebuild the table based on the calculated number of rows and columns.

11. The method for identifying the structure of a paper hydrological yearbook table according to claim 10, characterized in that: The traversing of all detected horizontal lines and vertical lines also includes filtering the detected straight lines based on the following rules: Length-based filtering: set a minimum length threshold to filter out lines whose length is less than the threshold; Intersection-based filtering: only keep lines that intersect with other lines; Range-based filtering: only keep lines that are within the table boundaries.

12. The method for identifying the structure of a paper hydrological yearbook table according to claim 1, characterized in that: The compliance check of the hydrological yearbook data on the filled Excel table includes multiple or all of the following checks: For the daily precipitation table, verify whether all cells in the value area are legal characters, verify whether the precipitation is consistent with the corresponding lunar phase in the statistical value area, verify whether the number of precipitation days is correct, verify whether the maximum daily amount is consistent with the value area, and verify whether the total of the annual statistics is consistent with the total value area; For the maximum precipitation table for each time period, verify whether all characters are legal characters, verify whether the station name exists, and verify whether the long-term precipitation is greater than or equal to the short-term maximum precipitation.

13. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores computer instructions, and the processor executes the method for identifying the structure of a paper hydrological yearbook table according to any one of claims 1 to 12 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that: The storage medium stores computer instructions, and when the computer instructions are executed by a computer, the method for identifying the structure of a paper hydrological yearbook table according to any one of claims 1 to 12 is executed.

Images