Table identification data synthesis method and device based on webpage rendering

Abstract

The invention discloses a method and a device for synthesizing table identification data based on webpage rendering. The method comprises the following steps of: randomly generating the size of a table; randomly generating the number, position and merging span of merging cells; generating style parameters of the table, wherein the style parameters comprise a frame style, a frame color, a background color, a font size, a font color and an alignment mode; using a GPT4 model to generate a table filling text which is matched with the table structure and accords with normal semantics; generating an HTML representation file and a CSS style file of the table; generating a JavaScript file, configuring screenshot parameter setting required after HTML front-end rendering, and obtaining function logic of coordinate frame positions of all table cell elements represented based on an HTML language; and rendering the generated HTML file in a webpage browser, carrying out webpage screenshot by using the generated JavaScript script to obtain a synthesized table image, reserving coordinate frame position information of each cell in the table, and finally, combining the HTML file and the reserved coordinate frame position information of the cells to generate complete table identification annotation content. And finally, the system stores the labeling result as a JSON format file.

Description

Technical Field

[0001] The invention relates to a table recognition data synthesis method and device based on web page rendering, belonging to the field of computer technology software. Background Art

[0002] In the Internet era, the rapid development of information technology has led to the transmission and storage of massive amounts of data in the form of images on the Internet at all times. These image data often contain massive amounts of text content, which has put forward more and more complex requirements for the use of image data. As a structure that efficiently carries text information, tables often appear in various images containing information introductions. It is of great significance to identify the structure and cell content of tables in these table images.

[0003] Currently, the mainstream methods for table recognition tasks are designed based on deep learning. These methods are more dependent on training data. However, the annotation of training data for table recognition is usually complex and cumbersome, which brings great pressure on researchers of table recognition tasks to collect training data. Summary of the invention

[0004] In view of the above problems and difficulties, the present invention proposes a method and device for synthesizing table recognition data based on web page rendering. The present invention can automatically generate a large amount of table recognition data, including its complex annotation content, and can customize the table structure template type to generate complex and diverse table recognition data.

[0005] The first aspect of the present invention relates to a method for synthesizing table recognition data based on web page rendering, and the specific steps are as follows:

[0006] S101: randomly selecting a table structure template from a pre-defined table structure template library, and then the system randomly generates the size of the table, that is, the number of rows and columns of the table;

[0007] S102: Randomly generate the number of merged cells, the position of merged cells, and the merge span of merged cells;

[0008] S103: Generate style parameters of the table according to the set table template type, including border style (such as solid line, dotted line, line thickness), border color, background color, font size, font color, alignment, etc.;

[0009] S104: Based on the generated table structure, use the GPT4 model to generate a table filling text that matches the table structure and conforms to normal semantics;

[0010] S105: Generate an HTML representation file and a CSS style file of the table according to the generated table structure, table style parameters, and table fill text;

[0011] S106: Automatically generate a JavaScript script file according to the set template, configure the screenshot parameter settings required after HTML front-end rendering and obtain the function logic of the coordinate box position of each cell element of the table represented by the HTML language;

[0012] S107: Render the generated HTML file in a web browser, and use the generated JavaScript script to take a screenshot of the web page to obtain a synthesized table image, while retaining the coordinate box position information of each cell in the table. Finally, combine the HTML file and the retained cell coordinate box position information to generate a complete table recognition annotation content. Finally, the system saves the annotation results as a JSON format file.

[0013] Specifically, in step S101, a table structure template is randomly selected from a pre-defined table structure template library. The table structure template includes five table structure templates: a table without a header, a table containing only a row header, a table containing only a column header, a table containing both a row header and a column header, and a complex nested table.

[0014] Specifically, in step S103, the style parameters of the table are generated according to the set table template type, including border style (such as solid line, dotted line, line thickness), border color, background color, font size, font color, alignment, etc. The border style is set by the CSS attribute border-style; the border thickness is set by the CSS attribute border-width; the border color is set by the CSS attribute border-color; the background color is set by the CSS attribute background-color; the font size is set by the CSS attribute font-size; the font color is set by the CSS attribute color; and the alignment is set by the CSS attributes text-align and vertical-align.

[0015] Specifically, in step S104, based on the generated table structure, the GPT4 model is used to generate table filling text that matches the table structure and conforms to normal semantics. Specifically, the API interface of the OpenAI chatbot program is called, and the set table structure template type and the generated table structure are organized as prompt statements of the input model to guide the GPT4 model to generate semantically correct table filling content.

[0016] Specifically, in step S107, the Selenium WebDriver tool is used to simulate the user's operation in the browser. The system selects a suitable WebDriver driver according to the set browser type and version, and configures the browser's startup parameters and options to achieve the correct loading and rendering of the HTML file. The system calls the generated JavaScript script to take a screenshot of the rendered HTML page as a synthesized table image; the script simultaneously outputs the pixel-level coordinate position of each cell element of the table represented by the HTML language as the cell detection annotation of the table recognition data. Then the HTML content string representing the table is combined with the previously obtained cell coordinate box position information to form a complete table recognition data annotation content.

[0017] A second aspect of the present invention relates to a table recognition data synthesis device based on web page rendering, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a table recognition data synthesis method based on web page rendering of the present invention.

[0018] A second aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a table recognition data synthesis method based on web page rendering of the present invention.

[0019] The present invention realizes the automatic generation of a table by randomly selecting a table structure template, generating table size, setting style parameters and other steps; at the same time, the GPT4 model is used to generate table filling text that conforms to semantic logic, and generates HTML representation files and CSS style files; the system also automatically generates JavaScript scripts for web page screenshots and obtaining the location information of table cells; finally, the system renders the HTML file in the browser, generates a table image and retains the cell location information, and saves the annotation results in JSON format. This invention can automatically generate a large amount of table recognition data, and can also highly liberalize the definition of table structure template types to generate complex and diverse table recognition data, and has good application prospects.

[0020] In summary, the present invention creates a table recognition data synthesis system based on web page rendering, which has the following beneficial effects: (1) It can automatically generate a large amount of table recognition data, including its complex annotation content, saving a lot of labor time for annotating table recognition data. (2) It can customize the table structure template type to generate complex and diverse table recognition data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is the overall flow chart of the method of the present invention.

[0023] Figure 2 This is an example of the format of the annotation content of the table identification data synthesized by the present invention. Specific implementation methods

[0025] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0026] Example 1

[0027] This embodiment relates to a method for synthesizing table recognition data based on web page rendering, which includes the following specific steps:

[0028] S101: randomly selecting a table structure template from a pre-defined table structure template library, and then the system randomly generates the size of the table, that is, the number of rows and columns of the table;

[0029] S102: Randomly generate the number of merged cells, the position of merged cells, and the merge span of merged cells;

[0030] S103: Generate style parameters of the table according to the set table template type, including border style (such as solid line, dotted line, line thickness), border color, background color, font size, font color, alignment, etc.;

[0031] S104: Based on the generated table structure, use the GPT4 model to generate a table filling text that matches the table structure and conforms to normal semantics;

[0032] S105: Generate an HTML representation file and a CSS style file of the table according to the generated table structure, table style parameters, and table fill text;

[0033] S106: Automatically generate a JavaScript script file according to the set template, configure the screenshot parameter settings required after HTML front-end rendering and obtain the function logic of the coordinate box position of each cell element of the table represented by the HTML language;

[0034] S107: Render the generated HTML file in a web browser, and use the generated JavaScript script to take a screenshot of the web page to obtain a synthesized table image, while retaining the coordinate box position information of each cell in the table. Finally, combine the HTML file and the retained cell coordinate box position information to generate a complete table recognition annotation content. Finally, the system saves the annotation results as a JSON format file.

[0035] Specifically, in step S101, a table structure template is randomly selected from a pre-defined table structure template library. The table structure templates include five table structure templates: table without header, table with only row header, table with only column header, table with both row header and column header, and complex nested table.

[0036] Specifically, in step S103, the style parameters of the table are generated according to the set table template type, including border style (such as solid line, dotted line, line thickness), border color, background color, font size, font color, alignment, etc. The border style is set by the CSS attribute border-style; the border thickness is set by the CSS attribute border-width; the border color is set by the CSS attribute border-color; the background color is set by the CSS attribute background-color; the font size is set by the CSS attribute font-size; the font color is set by the CSS attribute color; and the alignment is set by the CSS attributes text-align and vertical-align.

[0037] Specifically, in step S104, based on the generated table structure, the GPT4 model is used to generate table filling text that matches the table structure and conforms to normal semantics. Specifically, the API interface of the OpenAI chatbot program is called to organize the set table structure template type and the generated table structure into prompt statements of the input model, guiding the GPT4 model to generate semantically correct table filling content.

[0038] Specifically, in step S107, the Selenium WebDriver tool is used to simulate the user's operation in the browser. The system selects a suitable WebDriver driver according to the set browser type and version, and configures the browser's startup parameters and options to achieve the correct loading and rendering of the HTML file. The system calls the generated JavaScript script to take a screenshot of the rendered HTML page as a synthesized table image; the script also outputs the pixel-level coordinate position of each cell element of the table represented by the HTML language as the cell detection annotation of the table recognition data. Then the HTML content string representing the table is combined with the previously obtained cell coordinate box position information to form a complete table recognition data annotation content.

[0039] Example 2

[0040] A second aspect of the present invention relates to a table recognition data synthesis device based on web page rendering, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the e-commerce table image recognition method based on deep learning of Example 1.

[0041] Example 3

[0042] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the e-commerce table image recognition method based on deep learning of Example 1.

[0043] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for synthesizing table recognition data based on web page rendering, characterized in that: The following steps are involved: S101: the system randomly selects a table structure template from a pre-defined table structure template library, and then the system randomly generates the size of the table, that is, the number of rows and columns of the table; S102: The system randomly generates the number of merged cells, the position of the merged cells, and the merge span of the merged cells; S103: The system generates style parameters of the table according to the set table template type, including border style, border color, background color, font size, font color, and alignment; S104: Based on the generated table structure, the system uses the GPT4 model to generate a table filling text that matches the table structure and conforms to normal semantics; S105: The system generates an HTML representation file and a CSS style file of the table according to the generated table structure, table style parameters, and table fill text; S106: the system automatically generates a JavaScript script file according to the set template, configures the screenshot parameter settings required after HTML front-end rendering and obtains the function logic of the coordinate box position of each cell element of the table represented by the HTML language; S107: The system renders the generated HTML file in a web browser, and uses the generated JavaScript script to take a screenshot of the web page to obtain a composite table image, while retaining the coordinate box position information of each cell in the table; finally, the HTML file and the retained cell coordinate box position information are combined to generate a complete table recognition and annotation content; finally, the system saves the annotation results as a JSON format file.

2. A method for synthesizing table recognition data based on web page rendering according to claim 1, characterized in that: In step S101, a table structure template is randomly selected from a pre-defined table structure template library; the table structure template includes five table structure templates: a table without a header, a table containing only a row header, a table containing only a column header, a table containing both a row header and a column header, and a complex nested table.

3. A method for synthesizing table recognition data based on web page rendering according to claim 1, characterized in that: In step S103, the style parameters of the table are generated according to the set table template type, including border style, border color, background color, font size, font color, and alignment; the border style is set by the CSS attribute border-style; the border thickness is set by the CSS attribute border-width; the border color is set by the CSS attribute border-color; the background color is set by the CSS attribute background-color; the font size is set by the CSS attribute font-size; the font color is set by the CSS attribute color; and the alignment is set by the CSS attributes text-align and vertical-align.

4. A method for synthesizing table recognition data based on web page rendering according to claim 1, characterized in that: In step S104, based on the generated table structure, the GPT4 model is used to generate a table filling text that matches the table structure and conforms to normal semantics. The set table structure template type and the generated table structure are organized as prompt statements of the input model to guide the GPT4 model to generate semantically correct table filling content.

5. A method for synthesizing table recognition data based on web page rendering according to claim 1, characterized in that: In step S107, the Selenium WebDriver tool is used to simulate the user's operation in the browser; the system selects a suitable WebDriver driver according to the set browser type and version, and configures the browser's startup parameters and options to achieve correct loading and rendering of the HTML file; the system calls the generated JavaScript script to take a screenshot of the rendered HTML page as a synthesized table image; The script also outputs the pixel-level coordinate position of each cell element of the table represented by the HTML language as the cell detection annotation of the table recognition data; then it combines the HTML content string representing the table with the cell coordinate box position information obtained previously to form a complete table recognition data annotation content.

6. A table recognition data synthesis device based on web page rendering, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, they are used to implement a table recognition data synthesis method based on web page rendering as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a table recognition data synthesis method based on web page rendering as described in any one of claims 1 to 5 is implemented.

Images