Medium-and-small-span bridge disease inspection table merging row and column content distribution method

Abstract

The invention provides a medium-and-small-span bridge disease inspection table merging row and column content distribution method, and belongs to the field of bridge structure health monitoring. The method comprises the following steps: collecting bridge detection evaluation data including a bridge disease inspection table; the recognition model comprises a network encoder, a decoder, a Cycle-Paging module and an analysis processing module, and the bridge detection evaluation data in the PDF format is converted into the bridge detection evaluation data in the image format; constructing a cell re-division module, and re-dividing each row and each column of the table by using the reference rows and columns; distributing a textbox to corresponding rows in a table content distribution module, and performing content filling on the newly divided cells of the combined rows and columns; and constructing an adjustment and output module, adjusting the sequence of each row of textboxes, and outputting a processed result. According to the method, the problems of content missing and dislocation of the extraction result caused by row and column combination are solved, the mining effect of historical disease information in the bridge disease check table is optimized, and the comprehensiveness and accuracy of information extraction are improved.

Description

Technical Field

[0001] The invention belongs to the field of bridge structure health monitoring, and in particular relates to a method for allocating content of merged rows and columns of a small and medium span bridge disease inspection list. Background Art

[0002] Small and medium-span bridges are widely distributed in my country. Unlike extra-large and large-span bridges, most small and medium-span bridges do not have dedicated large-scale structural health monitoring systems installed. The way to obtain their service status is mostly through regular inspections or special inspections specified in the maintenance specifications. Together with the data at the completion acceptance, they constitute the entire "health file" of small and medium-span bridges, which are also called bridge inspection and evaluation data. As an important part of bridge inspection and evaluation data, the bridge disease checklist is of great significance to the evolution of bridge diseases and the formulation of maintenance strategies. The information in the checklist can lay a data foundation for the establishment of a theoretical system for the maintenance of small and medium-span bridges, and provide a guarantee for the subsequent research on efficient maintenance strategies for large-scale and wide-ranging small and medium-span bridges.

[0003] The bridge defect inspection table contains the detailed conditions of the defects of each component in the bridge superstructure, substructure, and bridge deck system during previous inspections. The description of the defects of each structure and component will also appear in the bridge inspection and evaluation data, but these descriptions are relatively concise, and most of them only write the type of defective component and the type of defect, without elaborating on the specific components and conditions of the defects. The bridge defect inspection table records in detail the name, component number, defect location, defect type, defect characteristics, scale, and photo number of the defective component. In order to obtain the information in the bridge defect inspection table, it is necessary to establish an information extraction method for this table structure. However, compared with the extraction of defect description text, the extraction of the bridge defect inspection table is more difficult. Different inspection companies in different regions have different writing and recording methods for bridge inspection and evaluation data, which makes the formats of bridge defect inspection tables different. Some tables have special merged rows and columns. This table structure will affect the structured information extraction method, causing some extraction results to be missing or misplaced, affecting the final information expression. Therefore, how to effectively solve the information allocation problem of merging rows and columns in the process of structured extraction of tabular information, so as to accurately and efficiently extract bridge structure damage information and provide a data basis for bridge damage evolution research and maintenance strategy formulation, is a key technical problem that needs to be solved urgently. Summary of the invention

[0004] Based on the above shortcomings, the present invention provides a method for allocating the content of merged rows and columns in a small and medium-span bridge defect inspection table, in order to solve the problem that the existing table information structured extraction method cannot accurately extract the merged row and column information in the bridge defect inspection table, and thus cannot efficiently and accurately generate information suitable for research in the field of bridge structure health monitoring.

[0005] The technical solution adopted by the present invention is as follows: A method for allocating the merged row and column contents of the disease inspection form for medium and small span bridges, the steps are as follows:

[0006] Step S1: Collect the bridge inspection and assessment materials of each era containing the bridge disease inspection form, and the forms of the materials include handwritten materials, paper records, material record images, and material electronic documents, and convert them into PDF format;

[0007] Step S2: Construct an identification model: including a network encoder, a decoder, a Cycle-Pairing module for positioning cells and learning splicing, and a parsing and processing module for restoring the complete table structure information;

[0008] Step S3: Preprocess the bridge inspection and assessment materials in PDF format and convert them into image format with a resolution of 300 DPI, input the bridge inspection and assessment materials into the identification model, and output the information of the recognized text, text box coordinates, and table box coordinates;

[0009] Step S4: Construct a cell redivision module, and the specific steps are as follows:

[0010] Step S4.1: Define the first row of the table as the reference row and the last column as the reference column, and the cell coordinates of the table are in the form of polygon xmin , polygon xmax , polygon ymin , polygon ymax , traverse all cell coordinates, and define the cell coordinate polygon ymin The smallest cell as the column reference cell, and record the cell coordinate polygon ymin as fixed ymin , define the cell coordinate polygon xmax The largest cell as the row reference cell, and record the cell coordinate polygon xmax as fixed xmax ;

[0011] Step S4.2: When there is a cell polygon ymin and fixed ymin Satisfy |fixed ymin -polygon ymin |≤10, record the polygon of this cell xmin and polygon xmax as x min and x max , the recorded x of each cell min and x maxThat is the width range of each column. When there is a cell polygon xmax and fixed xmax satisfy |fixed xmax - polygon xmax | ≤ 10, record the polygon of this cell ymin and polygon ymax as y min and y max . The y min and y max of each recorded cell are the height ranges of each row. The area formed by the intersection of the widths of each column and the heights of each row is the newly re-divided cell;

[0012] Step S5: Construct a table content distribution module. The specific steps are as follows:

[0013] Step S5.1: Record the upper left corner coordinates among the four corner coordinates of each text box as x min1 , y min1 , and the lower right corner coordinates as x max1 , y max1 . Calculate y avg1 = (y min1 + y max1 ) / 2, x avg1 = (x min1 + x max1 ) / 2. Traverse each cell of the reference column. When the y avg1 of the text box satisfies y min ≤ y avg1 ≤ y max , that is, assign this text box to the row within the range of this y min , y max . Add the x avg1 , text, and y avg1 of the text box to the list of each row;

[0014] Step S5.2: Calculate the average row position row avg = (y min + y max ) / 2, and the average column position column avg = (x min + x max ) / 2. If there is no x avg1 and y avg1 of a certain text box in the list of each row that simultaneously satisfy x min ≤ x avg1 ≤ x max and y min ≤ y avg1 ≤ y max , find the one that simultaneously satisfies polygonxmin ≤ column avg ≤ polygon xmax and polygon ymin ≤ row avg ≤ polygon ymax of the cell;

[0015] Step S5.3: If the polygon of this cell xmin , polygon xmax , polygon ymin , polygon ymax simultaneously makes a text box satisfy polygon xmin ≤ x avg1 ≤ polygon xmax and polygon ymin ≤ y avg1 ≤ polygon ymax , then the x avg1 , text, y avg1 of this text box are added to the list of this row, completing the distribution of the content of merging rows and columns;

[0016] Step S6: Construct an adjustment and output module: Sort and adjust the allocated content according to the coordinates of the text boxes. Arrange the text content of each row in ascending order according to the x avg1 of the text boxes in each row. Traverse each text box. When the difference in x avg1 of the text boxes in the same row is less than 40, then these text boxes in this row are arranged in ascending order according to their y avg1 , completing the adjustment of the text in each row. Write the output result into a specified document to achieve the accurate extraction of the information of the bridge disease inspection form for merging rows and columns.

[0017] On the other hand, the object of the present invention is implemented by a computer system, which includes a processor and a computer-readable storage medium communicably connected to the processor. Computer instructions are stored in the computer-readable storage medium. When the computer instructions are executed by the processor, the steps of the method described above are implemented.

[0018] The beneficial effects and advantages of the present invention are as follows: The present invention solves the problems of missing and misplaced extraction results caused by merging rows and columns, and realizes the accurate and efficient extraction of disease information in the bridge disease inspection form. The present invention optimizes the mining effect of historical disease information in the bridge disease inspection form, improves the comprehensiveness and accuracy of the extracted information, and lays a data foundation for the establishment of the maintenance theory system of small and medium-span bridges. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1Frame diagram of the method for allocating the merged row and column contents of the disease inspection form for small and medium - span bridges proposed by the present invention;

[0020] Figure 2 Recognition effect diagram of the recognition model for bridge inspection reports;

[0021] Figure 3 Partial ocr_result diagram extracted only using the recognition model;

[0022] Figure 4 Partial polygons diagram extracted only using the recognition model;

[0023] Figure 5 Comparison diagram of the extraction results of the method for allocating the merged row and column contents of the disease inspection form for small and medium - span bridges proposed by the present invention and the extraction results without using this method. Specific implementation manners

[0024] The present invention will be further described in detail below in conjunction with embodiments and the accompanying drawings, but the content of the present invention is not limited thereto. Those skilled in the art can make modifications in form and details, and these modifications are within the protection scope of the present invention.

[0025] Embodiment 1

[0026] As Figure 1 shown, a method for allocating the merged row and column contents of the disease inspection form for small and medium - span bridges is as follows:

[0027] Step S1: The bridge inspection and evaluation data containing the disease inspection form for the bridge used in this example is the regular inspection report of a certain bridge in a certain city in 2021. The extracted disease inspection form for the bridge is the disease inspection form for the superstructure, and the report format is PDF.

[0028] Step S2: Build a recognition model on the computer: Use Hourglass - 104 as the network encoder, Deconv as the decoder, the Cycle - Pairing module to locate cells and learn stitching, and the parsing processing module to restore the complete table structure information;

[0029] Step S3: Convert each page of the PDF file into an image format, and the converted image resolution is 300 DPI. Run the table structure recognition model as Figure 2 、 3 、4 shown, and output information such as the four - corner coordinates of the text box, text content, confidence level, the upper - left and lower - right coordinates of the text box, and the upper - left and lower - right coordinates of each cell in the table can be obtained;

[0030] Step S4: Build a cell re - division module, and the specific steps are as follows:

[0031] Step S4.1: Define the first row header of the table as the reference row, the photo number in the last column as the reference column, and the table cell coordinates in the form of polygon xmin 、polygon xmax 、polygon ymin 、polygon ymax , traverse all cell coordinates, and define the cell coordinate polygon ymin with the smallest value as the column reference cell, and record this cell coordinate polygon ymin as fixed ymin , define the cell coordinate polygon xmax with the largest value as the row reference cell, and record this cell coordinate polygon xmax as fixed xmax ;

[0032] Step S4.2: When there is a cell polygon ymin and fixed ymin satisfy |fixed ymin - polygon ymin | ≤ 10, record the polygon xmin and polygon xmax of this cell as x min and x max , and the x min and x max of each recorded cell are the width ranges of each column. When there is a cell polygon xmax and fixed xmax satisfy |fixed xmax - polygon xmax | ≤ 10, record the polygon ymin and polygon ymax of this cell as y min and y max , and the y min and y max of each recorded cell are the height ranges of each row. The area formed by the intersection of the widths of each column and the heights of each row is the newly re-divided cell;

[0033] Step S5: Construct a table content distribution module, and the specific steps are as follows:

[0034] Step S5.1: Record the upper left corner coordinates x min1 、y min1 among the four corner coordinates of each text box, and the lower right corner coordinates x max1 、y max1 , calculate y avg1 = (y min1 + ymax1 ) / 2, x avg1 = (x min1 + x max1 ) / 2, traverse each cell in the reference column. When the text box y avg1 satisfies y min ≤ y avg1 ≤ y max , that is, assign the text box to the row where this y min , y max is located, and add the x avg1 , text, y avg1 of the text box to the list of each row;

[0035] Step S5.2: Calculate the average value of the row positions row avg = (y min + y max ) / 2, and the average value of the column positions column avg = (x min + x max ) / 2. If there is a situation where the x avg1 and y avg1 of a certain text box do not satisfy x min ≤ x avg1 ≤ x max and y min ≤ y avg1 ≤ y max at the same time in the list of each row (such as the merged cells in the columns of "Serial Number" and "Component Name" in this example), find the cells that satisfy polygon xmin ≤ column avg ≤ polygon xmax and polygon ymin ≤ row avg ≤ polygon ymax ;

[0036] Step S5.3: If the polygon xmin , polygon xmax , polygon ymin , polygon ymax of this cell simultaneously make a certain text box satisfy polygon xmin ≤ x avg1 ≤ polygon xmax and polygon ymin ≤ y avg1 ≤ polygon ymax , then the x avg1 , text, y avg1Add it to the list of this row (in this example, the first and second columns of the 1st, 3rd, 4th, and 5th rows of the table), and complete the distribution of the merged row and column content;

[0037] Step S6: Construct an adjustment and output module: Sort and adjust the allocated content according to the text box coordinates, and arrange the text content of each row in ascending order according to the x of each text box in the row. Traverse each text box. When the difference in x of the text boxes in the same row is less than 40, then these text boxes in this row are arranged in ascending order according to their y avg1 Arrange the text content of each row in ascending order. Traverse each text box. When the difference in x of the text boxes in the same row avg1 is less than 40, then these text boxes in this row are arranged in ascending order according to their y avg1 Arrange the text of each row in ascending order, and write the output result into a specified document to accurately extract the information of the bridge disease inspection form with merged rows and columns.

[0038] The extraction results of the method of the present invention are compared with the extraction results of the method without distributing the merged row and column content as Figure 5 shown. The method of the present invention has a good distribution effect on the merged cell content of the "component name" column, making the output information more complete. The average row content completeness rate of the extraction result (row content completeness rate = the number of existing text boxes in a row / the number of text boxes that should be in a row) can reach 97.6%. However, the extraction result without using the method of distributing the merged row and column content has information missing in the "component name" column, and the average row content completeness rate is 89.7%. This verifies that the method of the present invention can improve the accuracy and completeness of content extraction.

Claims

1. A method for merging rows and columns of a small and medium span bridge defect inspection list, characterized in that: Here are the steps: Step S1: collecting bridge inspection and assessment data of various eras including bridge disease inspection forms, including handwritten data, paper record data, data record images, and data electronic documents, and converting them into PDF format; Step S2: construct a recognition model: including a network encoder, a decoder, a Cycle-Pairing module for locating cells and learning splicing, and a parsing processing module for recovering complete table structure information; Step S3: pre-process the bridge inspection and evaluation data in PDF format and convert it into image format with an image resolution of 300 DPI, input the bridge inspection and evaluation data into the recognition model, and output the information of the recognized text, text box coordinates, and table box coordinates; Step S4: construct a cell redivision module, the specific steps are as follows: Step S4.1: Define the first row of the table as the base row, the last column as the base column, and the table cell coordinates in polygon format xmin 、polygon xmax 、polygon ymin 、polygon ymax , traverse all cell coordinates and define the cell coordinate polygon ymin The smallest cell is the column base cell, and the cell coordinates are polygon. ymin fixed ymin , define the cell coordinate polygon xmax The largest cell is the row base cell, and the cell coordinates are polygon. xmax fixed xmax ; Step S4.2: When there is a cell polygon ymin with fixed ymin satisfied|fixed ymin -polygon ymin |≤10, record the polygon of the cell xmin and polygon xmax For x min and x max , the x of each cell is recorded min and x max That is the width range of each column. When there is a cell polygon xmax with fixed xmax satisfied|fixed xmax -polygon xmax |≤10, record the polygon of the cell ymin and polygon ymax for y min and max , the y value of each cell is recorded min and max That is, the height range of each row, and the area formed by the intersection of each column width and each row height is the newly divided cell; Step S5: construct a table content allocation module, the specific steps are as follows: Step S5.1: Note that the coordinate of the upper left corner of each text box is x min1 ,y min1 , the coordinate of the lower right corner is x max1 ,y max1 , calculate y avg1 =(y min1 +y max1 ) / 2, x avg1 =(x min1 +x max1 ) / 2, traverse each base column cell, when the text box y avg1 Satisfy min ≤y avg1 ≤y max When the text box is assigned to the y min ,y max The row in the range, change the x of the text box avg1 ,text,y avg1 Add to the list of rows; Step S5.2: Calculate the average row of each row avg =(y min +y max ) / 2, the average value of each column position avg =(x min +x max ) / 2, if the x of a text box does not exist in each row list avg1 and avg1 At the same time, x min ≤x avg1 ≤x max With y min ≤y avg1 ≤y max , find the polygon that satisfies both xmin ≤column avg ≤polygon xmax and polygon ymin ≤row avg ≤polygon ymax cells; Step S5.3: If the polygon of the cell xmin 、polygon xmax 、polygon ymin 、polygon ymax At the same time, make a text box satisfy the polygon xmin ≤x avg1 ≤polygon xmax and polygon ymin ≤y avg1 ≤polygon ymax , then the x of the text box avg1 ,text,y avg1 Add to the list for that row to complete the assignment of the contents of the merged row and column; Step S6: Construct adjustment and output module: sort and adjust the allocated content according to the text box coordinates, and avg1 Arrange the text content of each line from small to large, traverse each text box, and when a text box x appears in the same row avg1 If the difference is less than 40, the text boxes in this row are set according to their y avg1 Arrange from small to large, complete the text adjustment of each line, write the output results into the specified document, and realize the accurate extraction of the bridge disease inspection table information of the merged rows and columns.

2. A computer system comprising a processor and a computer-readable storage medium communicatively connected to the processor, wherein the computer-readable storage medium stores computer instructions, characterized in that: When the computer instructions are executed by the processor, the steps of the method according to claim 1 are implemented.

Images