A method of structured conversion of a data table scan image

Abstract

The application relates to a structured conversion method of a data table scanning image, comprising the following steps: 1) obtaining a scanning image of a data table; 2) extracting text position information and table position information in the scanning image; 3) obtaining row and column position information of the text in the table according to the text position information and the table position information; 4) recognizing text recognition information in each cell in the table one by one according to the row and column position information; 5) reconstructing an electronic table document containing the text recognition information and the table position information; and 6) converting the electronic table document into structured data in the form of a dictionary. Compared with the prior art, the application has high accuracy, realizes automatic structured conversion, and is high in efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a structured conversion method of a data table scanning image. BACKGROUND

[0002] At present, information can be divided into two categories, one of which can be represented by numbers or unified structures, referred to as structured data, and the other of which cannot be represented by numbers or unified structures, such as text, images, sound, web pages, etc., referred to as unstructured data. At present, a large amount of unstructured data is stored by enterprises, which cannot follow standard data structures and is difficult to understand or mobilize to some extent, and cannot meet the growing application requirements. The current traditional structured conversion method is to extract features of unstructured data, recognize named entities and relationships, and construct data tables according to the attributes to be processed, which requires manual participation and is low in efficiency.

[0003] At the same time, enterprises store unstructured data in the form of scanned electronic spreadsheets. The current electronic spreadsheet logic extraction method is to obtain all tables in an electronic spreadsheet document (such as an excel table software) by a table recognition algorithm, analyze the layout, extract the content according to the analysis result, and perform corresponding conversion processing to obtain structured information. However, a considerable part of information materials are mainly in the form of scanned tables and photo tables, which are not as neat and clear as electronic spreadsheets, and may be inclined or misaligned. SUMMARY

[0004] The present application is to overcome the defects of the prior art and provide a structured conversion method of a data table scanning image, which is high in accuracy, realizes automatic structured conversion, and is high in efficiency.

[0005] The object of the present application can be achieved by the following technical solutions:

[0006] A structured conversion method of a data table scanning image can be used for structured conversion of a power system substation equipment test report, and includes the following steps:

[0007] 1) Obtain a scanning image of a data table;

[0008] 2) Extract text position information and table position information in the scanning image;

[0009] 3) Obtain row and column position information of the text in the table according to the text position information and the table position information;

[0010] 4) Identify text recognition information in each cell in each table according to the row and column position information;

[0011] 5) Reconstruct the spreadsheet document containing text recognition information and table location information;

[0012] 6) Convert spreadsheet documents into structured data in dictionary format.

[0013] Furthermore, the text location information includes the text location information and the border location information of the cell where the text is located.

[0014] Furthermore, the process of extracting the text location information includes:

[0015] The location information of text in scanned images is extracted using OCR deep learning algorithms;

[0016] The scanned image is binarized to obtain the border position information of the cell containing the text.

[0017] Furthermore, the table location information includes the coordinates of the intersection of the horizontal and vertical lines in the table;

[0018] The process of extracting the table location information includes:

[0019] The binarized scanned image is segmented using erosion and dilation operations to obtain the positions of the horizontal and vertical lines in the table, and then the coordinates of the intersection of the horizontal and vertical lines are obtained.

[0020] Further, step 6) includes:

[0021] For each table in the spreadsheet document, determine whether the rows and columns of the table are aligned. If they are, the table is considered a standard table; otherwise, it is considered a non-standard table.

[0022] The standard table is structured using keyword filling;

[0023] For non-standard tables, we first construct a keyword library and the hierarchical relationships of text information within the keyword library, extract the subordination relationships of text information within the non-standard tables, and then transform the text information into structured data in dictionary form based on the subordination relationships.

[0024] Furthermore, the structured conversion method for the scanned image of the data table includes:

[0025] Save the scanned image obtained in step 1), the spreadsheet document obtained in step 5), and the structured data obtained in step 6) to the database.

[0026] Furthermore, the structured conversion method for the scanned image of the data table includes:

[0027] It provides an interface for querying the database by developing an interface based on the RESTful protocol.

[0028] Furthermore, the structured conversion method for the scanned image of the data table includes:

[0029] Search the database for the corresponding spreadsheet document based on the file name;

[0030] Retrieve structured data from the database based on data key values.

[0031] Furthermore, the structured conversion method for the scanned image of the data table includes:

[0032] Receive a query command for existing data and generate a data list of all spreadsheet documents and structured data in the database.

[0033] Furthermore, the structured conversion method for the scanned image of the data table includes:

[0034] Set unread tags for spreadsheet documents and structured data in the database;

[0035] Receive unread data query commands and generate a data list of all unread spreadsheet documents and structured data in the database.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) Based on the table position information and text position information, this invention sorts the text information in each table and extracts the text information from each table to reproduce the text and table in the spreadsheet document, which solves the problem of text wrapping in the table in the initial image, avoids the defect of treating the wrapping text as two texts when recognizing OCR, and solves the misalignment problem of conventional OCR cross-line recognition.

[0038] (2) This invention combines cell text recognition and table style logic mining to achieve accurate recognition and automatic structured conversion of test reports with high efficiency;

[0039] (3) This invention provides users with interfaces for querying data and files through interface development based on the Restful protocol, so that users can obtain a list of files and data, as well as a list of unread files and data. Through interface development based on the Restful protocol, it provides users with interfaces for obtaining data and files, so that users can obtain the corresponding files or data by file name or data key value. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0041] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0042] A structured transformation method for scanned images of data tables, such as Figure 1 This includes the following steps:

[0043] 1) Obtain a scanned image of the data table;

[0044] 2) Extract text and table location information from the scanned image;

[0045] 3) Based on the text position information and table position information, obtain the row and column position information of the text in the table;

[0046] 4) Based on the row and column position information, identify the text recognition information in each cell of the table one by one;

[0047] 5) Reconstruct the spreadsheet document containing text recognition information and table location information;

[0048] 6) Convert spreadsheet documents into structured data in dictionary format.

[0049] Text recognition information includes Chinese characters, symbols, and characters.

[0050] Text position information includes the position of the text itself and the position of the border of the cell containing the text.

[0051] Step 2) involves the extraction of text location information, which includes:

[0052] The location information of text in scanned images is extracted using OCR deep learning algorithms;

[0053] The scanned image is binarized to obtain the border position information of the cell containing the text.

[0054] Table location information includes the coordinates of the intersection of horizontal and vertical lines in the table;

[0055] Step 2) involves the following process for extracting table location information:

[0056] Based on binarization, the scanned image is segmented using erosion and dilation operations to separate different tables in a scanned image, extract the positions of horizontal and vertical lines in these tables, and then obtain the coordinates of the intersection of the horizontal and vertical lines.

[0057] Step 3) By comparing the coordinates of these intersection points with the text position information, we can obtain the row and column position information of the text information in the original table.

[0058] Step 4) Based on the table position information and text position information, sort the text information in each table, and reproduce the text and table in the Word document by extracting the text information from each table one by one. This solves the problem of line breaks in the text within the table in the initial image and avoids the defect of treating line breaks as two separate texts during OCR recognition.

[0059] Step 6) includes:

[0060] For each table in the spreadsheet document, determine whether the rows and columns of the table are aligned. If they are, the table is considered a standard table; otherwise, it is considered a non-standard table.

[0061] For standard tables, keyword filling is used to structure them;

[0062] For non-standard tables, we first construct a keyword database and the hierarchical relationships of text information within the database. We then extract the subordination relationships of text information within the non-standard tables and transform the text information into structured data in dictionary form based on these relationships. An example of a non-standard table is shown in Table 1.

[0063] Table 1 Examples of Non-Standard Tables

[0064]

[0065] Save the scanned image obtained in step 1), the spreadsheet document obtained in step 5), and the structured data obtained in step 6) to the database.

[0066] Structured transformation methods for scanned images of data tables include:

[0067] It provides an interface for querying the database by developing an interface based on the RESTful protocol.

[0068] Structured transformation methods for scanned images of data tables include:

[0069] Search the database for the corresponding spreadsheet document based on the file name;

[0070] Retrieve structured data from the database based on data key values.

[0071] Receive a query command for existing data and generate a data list of all spreadsheet documents and structured data in the database.

[0072] Set unread tags for spreadsheet documents and structured data in the database;

[0073] Receive unread data query commands and generate a data list of all unread spreadsheet documents and structured data in the database.

[0074] This embodiment proposes a method for the structured conversion of scanned images of data tables, providing an automatic and efficient method for the structured conversion, data storage, and service of test reports, facilitating the digital management, structured storage, and fusion of multi-source heterogeneous data of data assets.

[0075] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for structured conversion of scanned images of data tables, characterized in that, Includes the following steps: 1) Obtain a scanned image of the data table; 2) Extract text and table location information from the scanned image; 3) Based on the text position information and table position information, obtain the row and column position information of the text in the table; 4) Based on the row and column position information, identify the text recognition information in each cell of the table one by one; 5) Reconstruct the spreadsheet document containing text recognition information and table location information; 6) Convert spreadsheet documents into structured data in dictionary format; Step 6) includes: For each table in the spreadsheet document, determine whether the rows and columns of the table are aligned. If they are, the table is considered a standard table; otherwise, it is considered a non-standard table. The standard table is structured using keyword filling; For non-standard tables, we first construct a keyword library and the hierarchical relationships of text information within the keyword library, extract the subordination relationships of text information within the non-standard tables, and then transform the text information into structured data in dictionary form based on the subordination relationships.

2. The method for structured conversion of scanned data table images according to claim 1, characterized in that, The text location information includes the text location information and the border location information of the cell where the text is located.

3. The method for structured conversion of scanned data table images according to claim 2, wherein the process of extracting text location information includes: The location information of text in scanned images is extracted using OCR deep learning algorithms; The scanned image is binarized to obtain the border position information of the cell containing the text.

4. The method for structured conversion of scanned data table images according to claim 3, characterized in that, The table location information includes the coordinates of the intersection of the horizontal and vertical lines in the table; The process of extracting the table location information includes: The binarized scanned image is segmented using erosion and dilation operations to obtain the positions of the horizontal and vertical lines in the table, and then the coordinates of the intersection of the horizontal and vertical lines are obtained.

5. The method for structured conversion of scanned data table images according to claim 1, characterized in that, include: Save the scanned image obtained in step 1), the spreadsheet document obtained in step 5), and the structured data obtained in step 6) to the database.

6. The method for structured conversion of scanned data table images according to claim 5, characterized in that, include: It provides an interface for querying the database by developing an interface based on the RESTful protocol.

7. The method for structured conversion of scanned data table images according to claim 5, characterized in that, include: Search the database for the corresponding spreadsheet document based on the file name; Retrieve structured data from the database based on data key values.

8. The method for structured conversion of scanned data table images according to claim 5, characterized in that, include: Receive a query command for existing data and generate a data list of all spreadsheet documents and structured data in the database.

9. The method for structured conversion of scanned data table images according to claim 5, characterized in that, include: Set unread tags for spreadsheet documents and structured data in the database; Receive unread data query commands and generate a data list of all unread spreadsheet documents and structured data in the database.

Images