Table image and table LaTeX code generation method and device

Abstract

The invention discloses a table image and table LaTeX code generation method and device, and the method comprises the steps: determining table dimension parameters, so as to generate LaTeX table creation codes corresponding to table dimensions; filling the LaTeX table creation code and the field to which the data belongs into a cue word template, inputting the cue word into a large language model, and outputting a new LaTeX table creation code; in the new LaTeX table creation code, replacing placeholders which originally represent the content of each cell with headers or corresponding data in the field to which the data belongs; in the new LaTeX table creation code, cells in the new LaTeX table creation code are randomly merged, and a table LaTeX code is obtained; and generating a table image corresponding to the table LaTeX code by using a rendering tool. According to the method, respective advantages of computer vision and a large language model are combined, and a plurality of modal information such as images, texts and the like are utilized, so that the 1t; a table image: a table LaTeX code gt; the large-scale automatic generation is of great significance in the fields of table recognition technology research, literature digitization and the like.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and device for generating a table image and a table LaTeX code. Background Art

[0002] Table recognition technology plays an important role in the digital transformation of various industries. This technology can automatically extract table structure and content from documents, thereby greatly improving data processing efficiency, reducing manual input errors, and accelerating the information analysis process. Table recognition technology has been widely used in many fields such as financial statement analysis, medical record organization, and data extraction from academic literature.

[0003] At present, there are many challenges in achieving efficient table recognition, the most notable of which is the collection and annotation of training data. Due to the diverse styles, complex formats, and sensitive information involved in tables, it is particularly difficult to obtain a large number of representative annotated datasets. In addition, tables in different fields have specific format requirements, which further increases the difficulty of building a universal and accurate table recognition model. To overcome these obstacles, researchers usually need to adopt data augmentation techniques, transfer learning methods, or develop new privacy protection measures to promote the development of this field. However, existing technologies still have difficulty in solving the problems of insufficient diversity of table structures and generation of domain-specific data when generating <table images, table LaTeX codes> in large quantities.

[0004] The present invention combines the domain-specific data generation capability of a large language model with the image generation capability of computer vision, and proposes a <table image, table LaTeX code> generation method, which has important application value for the research of high-precision table recognition algorithms and document digitization. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for generating a table image and a table LaTeX code, which solves the problems of insufficient table structure diversity and field-specific data generation faced by the prior art when generating the two at the same time.

[0006] The technical solution adopted by the present invention is as follows: A method for generating a table image and a table LaTeX code, comprising: Determine table dimension parameters to generate LaTeX table creation code corresponding to the table dimensions; Fill the LaTeX table creation code and the field to which the data belongs into a prompt word template, input the prompt word into a large language model, and output a new LaTeX table creation code; in the new LaTeX table creation code, the original placeholder representing the content of each cell will be replaced by the header or corresponding data of the field to which the data belongs; In the new LaTeX table creation code, cells therein are randomly merged to obtain a table LaTeX code; A rendering tool is used to generate a table image corresponding to the table LaTeX code.

[0007] Furthermore, according to the table image and the table LaTeX code, <table image, table LaTeX code> data is constructed; by changing the table dimension parameters and the field to which the data belongs, a data set suitable for table recognition algorithm research is generated.

[0008] Furthermore, the method further includes: inputting the table image and the table style image into an image style transfer model, and outputting a new table image.

[0009] Furthermore, <table image, table LaTeX code> data is constructed based on the new table image and the table LaTeX code; and a data set suitable for table recognition algorithm research is generated by changing the table dimension parameters and the field to which the data belongs.

[0010] Furthermore, the table-style image is randomly selected from a pre-constructed table-style image database.

[0011] Furthermore, the rendering tools include pdflatex, xelatex and lualatex.

[0012] The present invention also provides a table image and table LaTeX code generation device, comprising: A LaTeX table creation code module is used to generate a LaTeX table creation code according to an input table dimension parameter; fill the LaTeX table creation code and the field to which the data belongs into a prompt word template, input the prompt word into a large language model, and output a new LaTeX table creation code; in the new LaTeX table creation code, the original placeholder representing the content of each cell will be replaced by a table header or corresponding data of the field to which the data belongs; A layout adjustment module is used in the new LaTeX table creation code to randomly merge cells therein to obtain a table LaTeX code; The table image rendering module is used to generate a table image corresponding to the table LaTeX code using a rendering tool.

[0013] Furthermore, it also includes: The table image rendering module is used to input the table image and the table style image into the image style transfer model and output a new table image.

[0014] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned method for generating a table image and table LaTeX code.

[0015] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for generating a table image and a table LaTeX code is implemented.

[0016] Compared with the prior art, the beneficial effect of the present invention is that the present invention combines the specific domain data generation capability of the large language model and the image generation capability of computer vision, and then combines the image style transfer model to generate a diverse table filled with the required domain data, thereby solving the problem of insufficient diversity of table structure and generation of domain-specific data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0018] Figure 1 A schematic diagram of a method for generating a table image and table LaTeX code provided by an embodiment of the present invention; Figure 2 A schematic diagram of the effect after a layout adjustment provided by an embodiment of the present invention; Figure 3 A schematic diagram of an image style transfer process provided by an embodiment of the present invention; Figure 4 A schematic diagram of a module of a table image and table LaTeX code generation device provided by an embodiment of the present invention; Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0021] A table image and table LaTeX code generation method of the present invention, such as Figure 1 As shown, the following steps are included: (1) Determine the table dimension parameters and generate the LaTeX table creation code corresponding to the table dimensions; Specifically, a LaTeX table creation code with a fixed format is generated according to the input table dimension parameters. The characteristic of this LaTeX table creation code is that the table rendered is black text on a white background, does not contain cell merging, and does not contain any customization of the layout appearance, such as row height, column width, font, background color, cell border settings, etc. Among them, each cell content is filled with a placeholder. Among them, the table dimension parameter refers to the number of rows and columns of the table, such as 5 rows and 7 columns.

[0022] In one embodiment, the table dimension (number of rows, number of columns) will be input to a string processing module, so that it can modify the internal LaTeX table creation code template and output a LaTeX table creation code. The LaTeX table creation code template used in this embodiment is as follows, where the input table dimension parameters are used to control the "number of columns" and "number of rows" respectively, and the data content placeholder of each cell is an asterisk (*): \documentclass{article} \usepackage{array} \begin{document} \begin{tabular}{|*{number of columns}{c|}} {\hline *&*&*&*&*&*&*} × {number of rows} \hline \end{tabular} \end{document} (2) Filling the LaTeX table creation code and the field to which the data belongs into a prompt word template, inputting the prompt word into the large language model, and outputting a new LaTeX table creation code; in the new LaTeX table creation code, the original placeholder representing the content of each cell will be replaced by the table header or corresponding data of the field to which the data belongs; Specifically, the LaTeX table creation code generated in step (1) and the field to which the data belongs are filled into a large language model prompt word template in a fixed format to form a complete prompt word. The prompt word will be input into a large language model (such as QWen2.5, Llama3, Mistral Large, etc.), so that the large language model outputs a new LaTeX table creation code. In the new LaTeX table creation code, the original placeholders representing the contents of each cell will be replaced with the table header or corresponding data of the field to which the data belongs, and the other contents of the original LaTeX table creation code remain unchanged. Among them, the field to which the data belongs refers to the field to which the data filled in the table belongs, such as the subject field: Earth Science.

[0023] In one embodiment, the prompt word template used is as follows: "You are a {data field} worker with rich LaTeX writing experience. The following is a LaTeX table creation code with {number of rows} rows and {number of columns} columns, where the content of each cell is represented by {placeholder}. Please fill in the data content of the subject {data field} in each cell. Note that the structure of the table should not be changed. The following is the LaTeX table creation code you need to fill in the data: {LaTeX table creation code}".

[0024] (3) In the new LaTeX table creation code, cells therein are randomly merged to obtain a table LaTeX code; Specifically, the layout of the new LaTeX table creation code is adjusted, that is, the cells are randomly merged, and the number and position of the merged cells are random to obtain the table LaTeX code.

[0025] In one embodiment, a plurality of cells in the generated LaTeX table creation code are merged by rewriting the generated LaTeX table creation code. Figure 2 The table image rendered by the LaTeX table creation code before and after the rewrite shows the effect of the rewrite.

[0026] The LaTeX code rewriting method for merging cells across columns in one embodiment is to use the \multicolumn method, as follows: \begin{tabular}{|c|c|c|} \hline Column 1&\multicolumn{2}{|c|}{Column 2 and Column 3 merged} \\ \hline Data 1&Data 2&Data 3 \\ \hline \end{tabular} The LaTeX code rewriting method for merging cells across rows in one embodiment is to use the \multirow method, as follows: \usepackage{multirow} \begin{tabular}{|c|c|c|} \hline \multirow{2}{*}{Row 1 and Row 2 merged}&Data 1&Data 2 \\ \cline{2-3} &Data 3&Data 4 \\ \hline Data 5&Data 6&Data 7 \\ \hline \end{tabular} (4) Generate a table image corresponding to the table LaTeX code using a rendering tool.

[0027] Specifically, a LaTeX rendering tool (such as pdflatex, xelatex or lualatex, etc.) is called to generate a table image corresponding to the table LaTeX code; the characteristic of the image is that it contains only one table, the table has black text on a white background, and all cells have borders.

[0028] In one embodiment, <table image, table LaTeX code> data is constructed based on the table image and the table LaTeX code; by changing the table dimension parameters and the field to which the data belongs, a data set suitable for table recognition algorithm research can be generated.

[0029] In one embodiment, if Figure 3 As shown, the method further includes step (5): inputting the table image and the table style image into an image style transfer model, and outputting a new table image.

[0030] Specifically, the table image generated in step (4) and a table style image are input into an image style transfer model to obtain a new table image. The image style transfer model used in this embodiment is StyleGAN2. The table style image is randomly selected from a pre-built table style image database, and its characteristic is that the cell contents of the table are all empty, and only the table appearance is retained. The characteristic of the final output table image is that its appearance will incorporate the characteristics of the table style image, and the possible changes include but are not limited to row height, column width, font, background color, alignment, cell border settings, etc.

[0031] In one embodiment, <table image, table LaTeX code> data is constructed based on the new table image and the table LaTeX code; by changing the table dimension parameters and the field to which the data belongs, a data set suitable for table recognition algorithm research can be generated.

[0032] The present invention also provides a table image and table LaTeX code generation device, such as Figure 4 As shown, including: A LaTeX table creation code module is used to generate a LaTeX table creation code according to an input table dimension parameter; fill the LaTeX table creation code and the field to which the data belongs into a prompt word template, input the prompt word into a large language model, and output a new LaTeX table creation code; in the new LaTeX table creation code, the original placeholder representing the content of each cell will be replaced by a table header or corresponding data of the field to which the data belongs; A layout adjustment module is used in the new LaTeX table creation code to randomly merge cells therein to obtain a table LaTeX code; The table image rendering module is used to generate a table image corresponding to the table LaTeX code using a rendering tool.

[0033] It should be noted that the device embodiment shown in this embodiment matches the content of the above method embodiment, and the content of the above method embodiment can be referred to, which will not be repeated here.

[0034] In one embodiment, the table image rendering module further includes: a module for inputting the table image and the table style image into an image style transfer model, and outputting a new table image.

[0035] It should be noted that the device embodiment shown in this embodiment also matches the content of the above method embodiment. The content of the above method embodiment can be referred to and will not be repeated here.

[0036] The present specification also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the above-mentioned method for generating a table image and a table LaTeX code.

[0037] This manual also provides Figure 5 A schematic structural diagram of an electronic device is shown in FIG. Figure 5 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned method for generating a table image and a table LaTeX code. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0038] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, an improvement to the circuit structure of diodes, transistors, switches, etc.) or a software improvement (an improvement to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and make a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0039] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0040] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0041] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0042] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0043] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0044] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0045] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0046] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0047] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0048] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0049] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0050] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0052] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0053] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for generating a table image and a table LaTeX code, characterized in that: include: Determine table dimension parameters to generate LaTeX table creation code corresponding to the table dimensions; Fill the LaTeX table creation code and the field to which the data belongs into a prompt word template, input the prompt word into a large language model, and output a new LaTeX table creation code; in the new LaTeX table creation code, the original placeholder representing the content of each cell will be replaced by the header or corresponding data of the field to which the data belongs; In the new LaTeX table creation code, cells therein are randomly merged to obtain a table LaTeX code; A rendering tool is used to generate a table image corresponding to the table LaTeX code.

2. A method for generating a table image and table LaTeX code according to claim 1, characterized in that: According to the table image and the table LaTeX code, <table image, table LaTeX code> data is constructed; by changing the table dimension parameters and the field to which the data belongs, a data set suitable for table recognition algorithm research is generated.

3. A method for generating a table image and table LaTeX code according to claim 1, characterized in that: Also includes: The table image and the table style image are input into an image style transfer model, and a new table image is output.

4. A method for generating a table image and table LaTeX code according to claim 3, characterized in that: According to the new table image and the table LaTeX code, <table image, table LaTeX code> data is constructed; by changing the table dimension parameters and the field to which the data belongs, a data set suitable for table recognition algorithm research is generated.

5. A method for generating a table image and table LaTeX code according to claim 3, characterized in that: The table-style image is randomly selected from a pre-constructed table-style image database.

6. A method for generating a table image and table LaTeX code according to claim 1, characterized in that: The rendering tools include pdflatex, xelatex and lualatex.

7. A table image and table LaTeX code generation device, characterized in that: include: LaTeX table creation code module, used to generate LaTeX table creation code according to the input table dimension parameters; Fill the LaTeX table creation code and the field to which the data belongs into a prompt word template, input the prompt word into a large language model, and output a new LaTeX table creation code; in the new LaTeX table creation code, the original placeholder representing the content of each cell will be replaced by the header or corresponding data of the field to which the data belongs; A layout adjustment module is used in the new LaTeX table creation code to randomly merge cells therein to obtain a table LaTeX code; The table image rendering module is used to generate a table image corresponding to the table LaTeX code using a rendering tool.

8. A table image and table LaTeX code generating device according to claim 7, characterized in that: Also includes: The table image rendering module is used to input the table image and the table style image into the image style transfer model and output a new table image.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement a table image and table LaTeX code generation method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for generating a table image and a table LaTeX code as described in any one of claims 1 to 6 is implemented.

Images