Method and system for converting complex structure table image into Excel file

Abstract

The invention discloses a method and system for converting an Excel file to a complex structure table image, and the method comprises the steps: obtaining image data, carrying out the feature extraction of the image data through a first backbone network, and obtaining feature information; performing first transformation on the feature information to obtain first feature information, and inputting the first feature information to a first branch to obtain positioning information; inputting the first feature information to a second branch based on the positioning information to obtain a first sequence; replacing the first sequence according to the positioning information and the first sequence to obtain a first code; and the first code is converted by utilizing a first tool to obtain the Excel file, and the Excel file has better performance for table images with complex structures.

Description

Technical Field

[0001] The present invention relates to the technical field of document image conversion, and in particular to a method and system for converting Excel files oriented to complex structure table images. Background Art

[0002] With the rapid development of digital media, network and hardware technology, the number of images generated in office scenes has increased sharply, and image data has become an increasingly important source of work data. A large number of paper documents, especially table documents, are in urgent need of digitization for archiving and convenient retrieval. How to quickly extract the required information from the complex images has become a key challenge. In an office environment, extracting information from table images is a common task, such as invoice reimbursement entry, medical insurance reimbursement verification, and cargo inventory. However, this work is often tedious and error-prone, resulting in high labor costs. One of the key challenges in the field of document image recognition is the accurate recognition of table structure. As a special type of document image, the structure of the combination of rows and columns and the characteristics of cell merging in table images bring significant difficulties to the integrated processing of document images, so it has become a research hotspot and difficulty in the current industry and academia. Table image recognition technology is of great significance in data simplification, office process optimization, and information and intelligent development. The specific statement is as follows: Convert data format and reduce storage space. Reconstructing the content of table images and converting them from image data to structured sequence text data can significantly reduce the amount of data storage. Taking the common VAT invoice image as an example, 10,000 images in JPEG compression format with a resolution of 1059×693 pixels have a storage size of about 1.70GB. After the image is reconstructed and converted into a spreadsheet, only key text information is stored, and the storage size is only about 240MB, reducing the data storage volume by about 80%; portable office process. With the advent of the information age, whether it is enterprises, government agencies or individuals, a large number of documents such as contracts, reports, invoices, receipts, etc. will be generated. The popularity of smart phones and digital cameras has made it a convenient and common practice to convert documents into digital images. Compared with traditional questionnaire-style data filling, shooting or scanning images has the characteristics of simple operation, fast and efficient, and suitable for all ages, which is more in line with modern office needs. Converting document images into searchable electronic formats can significantly improve the efficiency and accuracy of information search and retrieval, thereby improving work efficiency and decision-making quality; realizing business intelligence. Unstructured image data usually needs to be preprocessed and feature extracted to extract valuable information from it and convert it into a structured form. Convolutional Neural Network (CNN) has shown strong capabilities in the field of image processing and can be used for tasks such as feature extraction, target detection, and image classification. Through CNN processing, image information can be effectively converted into structured data, which makes it easier to conduct subsequent data analysis, mining, and application. In downstream tasks related to tabular images, structured data can significantly improve processing efficiency and simplify operational processes, such as analyzing the financial statements of listed companies, counting the inflow and outflow of goods, and extracting paper invoice information.

[0003] The existing technology has problems such as being unable to recognize complex table images (with merged cells, cells with large cross-rows, etc.) with high accuracy, requiring numerous post-processing steps and high computing resource requirements. The technology of this application designs a lightweight ConvStem backbone network and a dual-branch detection and recognition strategy that can be embedded in any OCR tool. The entire model realizes the end-to-end conversion of table images into HTML / Excel / Latex code. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method and system for converting Excel files for complex structure table images to solve the problems of the prior art that complex table images (with merged cells, cells spanning large rows, etc.) cannot be recognized with high accuracy, numerous post-processing steps are required, and high computing resource requirements are required.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for converting an Excel file into an image of a complex structure table, comprising:

[0008] Acquire image data, and perform feature extraction on the image data using a first backbone network to obtain feature information;

[0009] Performing a first transformation on the feature information to obtain first feature information, and inputting the first feature information into a first branch to obtain positioning information;

[0010] Based on the positioning information, the first feature information is input into the second branch to obtain a first sequence;

[0011] replacing the first sequence according to the positioning information and the first sequence to obtain a first code;

[0012] The first code is converted using a first tool to obtain an Excel file.

[0013] As a preferred solution of the method for converting Excel files for complex structure table images described in the present invention, wherein: inputting the first feature information into the second branch comprises:

[0014] The first backbone network and the encoding head form a first encoder, and the image is input to the first encoder;

[0015] The information output by the first encoder is embedded into the first vocabulary.

[0016] As a preferred solution of the method for converting Excel files to complex structure table images described in the present invention, it also includes:

[0017] Performing a first decoding on the information embedded in the first vocabulary to obtain a feature tensor;

[0018] The feature tensor is mapped to the vocabulary space through a linear layer to generate the first sequence of outputs.

[0019] As a preferred solution of the method for converting Excel files for complex structure table images described in the present invention, wherein: inputting the first feature information into the first branch comprises:

[0020] Weighting and integrating the key domain information of the table to obtain characteristic data of the key domain information;

[0021] The feature data of key domain information is early fused and image dimensions are matched.

[0022] As a preferred solution of the method for converting Excel files to complex structure table images described in the present invention, it also includes:

[0023] Use the convolution kernel to sample the feature data, and use the set convolution kernel output and the segmentation map;

[0024] The positioning information is obtained according to the segmentation map, the positioning information is input into the first recognition tool, and the output text is stored.

[0025] As a preferred solution of the method for converting Excel files to complex structure table images of the present invention, the first decoding includes:

[0026] The target sequence is embedded by position and fused with the output features of the first encoder to obtain the input of the first decoder;

[0027] The fused feature and mask tensors are input into the first decoder to obtain the feature tensor.

[0028] As a preferred solution of the method for converting Excel files to complex structure table images described in the present invention, the first word table embedding includes:

[0029] Perform word embedding operation on each word in the target vocabulary, and perform position embedding operation on the sequence reference after word embedding;

[0030] For each position and dimension, if the dimension is even, the value of the position embedding is calculated using the sine function;

[0031] If the dimension is an odd number, the cosine function is used to calculate the value of the position embedding;

[0032] Add the word embedding and the position embedding to get the information after the first vocabulary embedding.

[0033] In a second aspect, the present invention provides a system for converting Excel files for complex structure table images, comprising:

[0034] A feature extraction module acquires image data, and performs feature extraction on the image data using a first backbone network to obtain feature information;

[0035] a transformation module, configured to perform a first transformation on the feature information to obtain first feature information, and input the first feature information into a first branch to obtain positioning information;

[0036] An input module, configured to input the first feature information into a second branch based on the positioning information to obtain a first sequence;

[0037] a replacement module, configured to replace the first sequence according to the positioning information and the first sequence to obtain a first code;

[0038] The conversion module is used to convert the first code using a first tool to obtain an Excel file.

[0039] In a third aspect, the present invention provides a computing device, comprising:

[0040] Memory and processor;

[0041] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for converting Excel files for complex structure table images are implemented.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for converting Excel files for complex structure table images.

[0043] Compared with the prior art, the present invention has the following beneficial effects: the present invention realizes the whole task chain of table image recognition, namely table detection, table structure recognition, and table content recognition, and can be converted into an actually usable and editable Excel file, which can effectively promote the automation and intelligence of business; it can achieve commercial-grade high precision, and on the most widely used PubTabNet and TableBank data sets (more than 100,000 images), the average recognition accuracy rate exceeds 85% (TDES index), especially for table images with complex structures; it requires few computing resources and adopts a lightweight backbone network design, with the backbone having only 24.08 million parameters. Even if the overall network model has only 100 million parameters, it can be deployed on a single machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0045] Figure 1 A schematic diagram of the overall process logic of a method for converting an Excel file to a complex structure table image according to an embodiment of the present invention;

[0046] Figure 2 A structural diagram of a table structure recognition model of a ConvStem backbone and an encoder of a method for converting an Excel file to a complex structure table image according to an embodiment of the present invention;

[0047] Figure 3 A demonstration diagram of the recognition effect of Chinese statistical table images with header merging in a method for converting Excel files to complex structure table images according to an embodiment of the present invention;

[0048] Figure 4 This is a demonstration diagram of the recognition effect of a large table image with multiple rows and columns merged in a method for converting Excel files to complex structure table images described in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0050] Example 1

[0051] Reference Figure 1-Figure 2 , as an embodiment of the present invention, provides a method for converting Excel files into complex structure table images, comprising:

[0052] S100: Acquire image data, and perform feature extraction on the image data using a first backbone network to obtain feature information;

[0053] In an optional embodiment, the first backbone network may include a YOLO backbone network, and the feature extraction of the image data using the YOLO backbone network includes identifying the cells in the table by training the YOLO model, predicting the position and size of each cell, and the features extracted by YOLO during the detection process are used for subsequent text recognition or structural analysis, and combined with other networks to achieve end-to-end training from image to structured data; it may also include a convolutional neural network backbone network, and the feature extraction of the image data using the convolutional neural network backbone network includes extracting the features of the table image and identifying the structure and content of the table;

[0054] In an optional embodiment, the first backbone network may also include a ResNet backbone network. Using the ResNet backbone network to extract features from image data includes: ResNet's feature extraction capability can be used to identify complex patterns and structures in table images. By increasing the depth of the network, ResNet can learn more abstract features. The ResNet pre-trained model can be used for transfer learning to migrate features learned from large-scale data sets to table image recognition tasks.

[0055] In the embodiment of the present application, the first backbone network includes:

[0056] like Figure 2 As shown in the figure, CovnStem is used as the backbone network, and the table information is sensed to process the image information of the document table to obtain multi-level feature information Finter, and a multi-layer strided convolution stem is designed as the ConvStem backbone network. The ConvStem backbone is designed as a multi-layer 3×3 convolution module combined with a normalization layer (BatchNorm), an activation layer (Activation) and other structures, and finally transformed with a 1×1 convolution to match the d-dimensional features of the Transformer;

[0057] Specifically, the document table image data information is [H, W, D], which represents the height, width, and depth of the input image respectively;

[0058] It should be noted that the ConvStem backbone network adopts a lightweight design. While maintaining efficient performance, it requires less computing resources, which is particularly important for deploying models in resource-constrained environments. Through multi-layer 3×3 convolution modules, ConvStem can extract multi-level feature information (Finter) of the image, which helps to capture the subtle structure and patterns in the table image. The combination of BatchNorm (normalization layer) and Activation (activation layer) helps the network learn more stable feature representations, while increasing nonlinearity, enabling the network to learn more complex function mappings. The final 1×1 convolution transformation ensures that the feature dimension of the ConvStem output matches the input dimension of the Transformer, which helps to achieve an end-to-end table image recognition process. The ConvStem backbone network can adapt to input images of different sizes. By processing data in the [H, W, D] dimensions, it can flexibly process table images of different resolutions.

[0059] S200: Perform a first transformation on the feature information to obtain first feature information, and input the first feature information into a first branch to obtain positioning information;

[0060] S300: Based on the positioning information, input the first feature information into the second branch to obtain a first sequence;

[0061] S400: replacing the first sequence according to the positioning information and the first sequence to obtain a first code;

[0062] S500: Convert the first code using a first tool to obtain an Excel file.

[0063] In an optional embodiment, the first tool may include Table2Excel, which analyzes the structure of the HTML table, captures the row, column and cell data of the table, and formats the data into a format recognizable by Excel. The user can trigger the conversion process by calling functions provided by Table2Excel, such as convertToExcel, and passing in the ID of the HTML table or the table element itself; the first tool may also include SheetJS, which parses the HTML table data, creates a new workbook, fills the table data into the workbook, and saves the workbook in Excel file format.

[0064] In an optional embodiment, the first tool may also include TableExport, which triggers the export operation by monitoring click events on the table. When the user clicks a button or a link, TableExport captures the table data and converts it into an Excel file.

[0065] In the embodiment of the present application, the first tool includes: js-export-excel;

[0066] Introduce the script file of js-export-excel library into HTML page, prepare HTML table data, ensure the table structure is clear and the data format is correct. Through JavaScript code, you can add event listeners to export buttons or links to trigger the export operation when the user clicks. Use the functions provided by js-export-excel to pass in the ID of HTML table or the table element itself, as well as the name of the export file, analyze the passed in HTML table, extract the row, column and cell data of the table, and convert the extracted data into Excel file format.

[0067] It should be noted that using js-export-excel as a tool to achieve HTML to Excel conversion provides an intuitive user interface that allows users to export tabular data by clicking a button, enhancing the user experience, automating the conversion process from HTML to Excel, reducing the need for manual copying and pasting of data, improving efficiency, ensuring the integrity and accuracy of data during the conversion process, and avoiding errors that may be introduced by manual operations.

[0068] It should be noted that the full task chain processing from table images to Excel files is realized, including table detection, structure recognition and content recognition, which promotes the automation and intelligence of business processes. It adopts a lightweight ConvStem backbone network design with a backbone network parameter of only 24.08 million and the overall model parameters controlled within 100 million, so that the model can be deployed on a single machine, reducing hardware requirements. By converting table images into structured sequential text data, the data storage volume is significantly reduced.

[0069] In the embodiment of the present application, the above step S200 includes the following sub-steps A1-A10;

[0070] In A1: weight and normalize the key field information in the table to obtain the characteristic data of the key field information;

[0071] In A2: early fusion of feature data of key domain information is performed and image dimensions are matched.

[0072] In A3: the feature data is sampled using a convolution kernel, and the segmentation map is mapped using the set convolution kernel output;

[0073] In A4: obtaining positioning information according to the segmentation map, inputting the positioning information into the first recognition tool, and storing the output text.

[0074] In an optional embodiment, the first recognition tool may include Baidu OCR, which integrates Baidu OCR service into the system, and sends the table image to Baidu's server for text recognition through API calls. The recognized text data can be returned and used for subsequent table structure reconstruction and data filling. The first recognition tool may also include iFlytek OCR, which integrates iFlytek OCR into the system, and calls its service through an API interface for text recognition. The recognized text can be integrated into the table data to complete the conversion of the table image to structured data.

[0075] In an optional embodiment, the first recognition tool may also include self-deployed OCR, which is tightly integrated with the table image recognition system to achieve end-to-end automated processing.

[0076] In the embodiment of the present application, the first recognition tool includes: any OCR;

[0077] Input the cell location information to OCR and store the output text in the dictionary. The model recognizes from top to bottom and from left to right, so the index of the dictionary is the sequential ID of the cell. Input the cell location information to OCR and store the output text in the dictionary.

[0078] Specifically, after the output of the backbone network is first transformed into a dimension transformation, it is input into a first branch as a cell positioning branch, which implements the acquisition of the location information of the cell;

[0079] Use axis attention to weight key domain information of the table, including rows, columns, and cells;

[0080] For example, row-wise attention performs linear layer, binarization, normalization and other processing on the feature values ​​on the X-axis to highlight the quantitative information of the "row" structure in the image.

[0081] Specifically, the feature data of rows, columns, and cells are fused early to match the image dimensions, which can be expressed as:

[0082] f Spatial =Concat(f cell ,f row ,f coll )

[0083] Among them, f cell ,f row ,f coll They are the feature data of row, column and cell respectively, and Concat is early fusion;

[0084] The convolution kernel is further sampled to 64, and the predicted segmentation map is obtained using the set 1×1 convolution kernel output, which is expressed as:

[0085]

[0086] Among them, K is the predicted classification type, which is "background", "boundary" and "cell". is a set of real numbers, H is the height of the input image, and W is the width of the input image;

[0087] The recognized cells are stored in the form of a dictionary Indicates the X value, Y value, width, and height of the center point of the cell respectively.

[0088] It should be noted that in addition to using OCR tools, this function can be embedded directly in the model, but it will increase the size of the model and reduce accuracy. The axis attention mechanism is used to weight the features of the rows, columns, and cells to improve the accuracy of table structure recognition, especially when processing complex tables. By storing the recognized text in a dictionary and using the sequential ID of the cell as an index, a structured output can be generated to facilitate subsequent data processing and analysis. The feature data of the rows, columns, and cells can be fused early to more effectively match the image dimensions and improve the efficiency of feature processing. The convolution kernel is used to sample to 64, and then predicted through a 1×1 convolution kernel to reduce the amount of calculation while maintaining recognition accuracy.

[0089] In the embodiment of the present application, the above steps S300 and S400 include the following sub-steps B1-B6;

[0090] In B1: the first backbone network and the encoding head are combined into a first encoder, and the image is input to the first encoder;

[0091] In B2: embedding the information output by the first encoder into the first vocabulary;

[0092] In B3: word embedding operation is performed on each word in the target vocabulary, and the sequence reference position embedding operation is performed on the word embedded;

[0093] In B4: For each position and dimension, if the dimension is even, the value of the position embedding is calculated using the sine function;

[0094] In B5: If the dimension is an odd number, the cosine function is used to calculate the value of the position embedding;

[0095] In B6: the word embedding and the position embedding are added to obtain the information after the first vocabulary embedding.

[0096] Specifically, after the output of the backbone network is transformed into a dimension transformation, it is input into the second branch as a sequence recognition branch, which converts the image information into an HTML sequence through a visual encoder, word embedding, text decoder, etc.

[0097] The ConvStem backbone network combines a 1×1 layer and an encoder (encoding head) to form a visual encoder with an output channel of 192, a downsampling factor of 8, a convolution kernel of 5*5, 4 encoding layers, 4 decoding layers, and an input image size of 504*504.

[0098] The first word list embedding includes embedding the information output by the visual encoder into the word list, and embedding the label word list data using the position embedding layer and Token Embed (token embedding), which is expressed as:

[0099] T pos =PositionEmbedding(TokenEmbedding(Y))

[0100] Among them, Y is the target HTML vocabulary, PositionEmbedding is the position embedding operation, and the output target sequence T pos ;

[0101] Position embedding is used to introduce position information into the sequence model so that the model can perceive the position of each element in the input sequence. The definition of position embedding is expressed as:

[0102]

[0103] Here, pos is the position in the sequence, n is the index in the embedding dimension, and d is the size of the embedding dimension.

[0104] In the embodiment of the present application, after completing steps B1-B6 in the above steps S300 and S400, the following steps B7-B10 are also included;

[0105] In B7: performing a first decoding on the information embedded in the first vocabulary to obtain a feature tensor;

[0106] In B8: the feature tensor is mapped to the vocabulary space through a linear layer to generate the first output sequence;

[0107] In B9: the target sequence is positionally embedded and fused with the output features of the first encoder to obtain the input of the first decoder;

[0108] In B10: the fused feature and mask tensors are input into the first decoder to obtain a feature tensor.

[0109] In an optional embodiment, the first decoder may include a recurrent neural network (RNN) first decoder, the text sequence recognized by OCR is input into the RNN, the RNN learns the context information of the text sequence through its hidden state, and thus generates or predicts the final text output; the first decoder may also include an autoencoder, the features extracted by the encoder are input into the first decoder, and the first decoder attempts to reconstruct the original table image or generate an HTML representation of the table.

[0110] In an optional embodiment, the first decoder may also include a convolutional neural network (CNN) first decoder, which inputs the feature map extracted by the CNN encoder into transposed convolutional layers, which gradually increase the spatial dimensions of the feature map and ultimately generate an output that matches the size of the original input image;

[0111] In the embodiment of the present application, the first decoder includes: a Transformer first decoder;

[0112] The information after word list embedding is input into Transformer decoding; the target sequence T after position embedding is added pos Together with the output M features, the decoded output is obtained through the Transformer decoder, which is expressed as:

[0113] D=Decoder(T pos ,M,T mask ,T pad_ask )

[0114] Among them, T mask is the mask tensor corresponding to the target sequence, which is used to prevent the model from seeing future words during decoding. This mask ensures the causal relationship of the prediction. pad_ask To mask the padding position in the target sequence, when processing variable-length sequences, in order to make the length of all sequences consistent, padding symbols are added to the end of the short sequences;

[0115] Specifically, the output feature tensor D is mapped to the vocabulary space through a linear layer (generator) to generate the final output text sequence represented as:

[0116] O=Linear(D)

[0117] In order to write the subsequent content, insert the "placeholder" string into the first sequence of HTML sequence to serve as the content placeholder;

[0118] The text dictionary output by the fusion positioning branch is looped and the first replacement is performed, including replacing the "placeholder" with the corresponding text in the dictionary, and outputting the first code with a complete table structure and text content as HTML.

[0119] It should be noted that the Transformer decoder is good at processing sequence data and can capture long-distance dependencies. By using mask tensors, causality in the decoding process is ensured, that is, when predicting the current word, the model does not see future words. The feature tensor is mapped to the vocabulary space through a linear layer to generate the final output text sequence, which helps to accurately reconstruct the text content in the table.

[0120] The above is a schematic scheme of a method for converting Excel files from complex structure table images in this embodiment. It should be noted that the technical scheme of the system for converting Excel files from complex structure table images and the technical scheme of the method for converting Excel files from complex structure table images described above belong to the same concept. For details not described in detail in the technical scheme of the system for converting Excel files from complex structure table images in this embodiment, please refer to the description of the technical scheme of the method for converting Excel files from complex structure table images described above.

[0121] Example 2

[0122] Reference Figure 3-Figure 4 This embodiment is different from the first embodiment. It provides an application embodiment of a method for converting Excel files into complex structure table images, in order to verify and illustrate the technical effects adopted in this method.

[0123] Take the Chinese statistical table image with merged headers and the large table image with merged multiple rows and columns as examples;

[0124] This network model is designed with a dual-branch strategy. The first branch is the table cell location branch, which detects the spatial position of the cell and segments the image in combination with OCR for text recognition. The second branch is the table structure recognition branch. After the features output by the backbone network enter the visual encoder, the image encoder is first used to extract features from the input image, and then the PositionalEmbed is used to embed the features. Finally, the Transformer encoder is used to encode the embedded features. Finally, the HTML structure tag sequence is obtained and combined with the table cell location branch output to recognize the text dictionary, realizing the task of reconstructing the table image content.

[0125] like Figure 3As shown in the figure, the model as a whole is also adaptable to Chinese tables. It accurately recognizes the row and column information of large tables. The detection model recognizes the existence of merged table headers. The structure recognition model accurately outputs the cross-column format in sequence generation. For complex statistical data in the table, the OCR tool recognizes statistical numbers, time formats, and financial currency symbols. However, due to the performance of the OCR tool, there are some erroneous outputs.

[0126] like Figure 4 As shown in the figure, the detection model recognizes the spatial significance of the blank area and assigns it row and column attributes in the structure sequence generation. For complex table images, the key is how to restore the regular row and column correspondence relationship. The model has a certain understanding of cross-row and cross-column text, and the final output content reconstruction fits the original image information well.

[0127] The dual-branch strategy can effectively handle complex table structures, including merged cells and multi-row and multi-column layouts, improving the accuracy of table structure recognition. The structure recognition model can accurately identify merged headers and output the correct cross-column format in sequence generation, which is crucial to maintaining the integrity of the table. The model can identify blank areas and assign row and column attributes to them, which helps to restore the regularity of the table, especially when processing irregular tables. For complex table images, the model can understand and restore the corresponding relationship between rows and columns, which is very important for the accurate reconstruction of the table content.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0129] Example 3

[0130] The system for converting Excel files into complex structure table images in this embodiment includes:

[0131] A feature extraction module acquires image data, and performs feature extraction on the image data using a first backbone network to obtain feature information;

[0132] a transformation module, configured to perform a first transformation on the feature information to obtain first feature information, and input the first feature information into a first branch to obtain positioning information;

[0133] An input module, configured to input the first feature information into a second branch based on the positioning information to obtain a first sequence;

[0134] a replacement module, configured to replace the first sequence according to the positioning information and the first sequence to obtain a first code;

[0135] The conversion module is used to convert the first code using a first tool to obtain an Excel file.

[0136] This embodiment further provides a computing device suitable for converting an Excel file for a complex structure table image, including:

[0137] A memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the method for converting Excel files for complex structure table images as proposed in the above embodiment.

[0138] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for converting Excel files for complex structure table images proposed in the above embodiment is implemented.

[0139] The storage medium proposed in this embodiment and the method for realizing the conversion of Excel files for complex structure table images proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0140] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.

Claims

1. A method for converting Excel files into complex structure table images, characterized in that: include: Acquire image data, and perform feature extraction on the image data using a first backbone network to obtain feature information; Performing a first transformation on the feature information to obtain first feature information, and inputting the first feature information into a first branch to obtain positioning information; Based on the positioning information, the first feature information is input into the second branch to obtain a first sequence; replacing the first sequence according to the positioning information and the first sequence to obtain a first code; The first code is converted using a first tool to obtain an Excel file.

2. The method for converting Excel files into complex structure table images as claimed in claim 1, characterized in that: Inputting the first feature information into the second branch comprises: The first backbone network and the encoding head form a first encoder, and the image is input to the first encoder; The information output by the first encoder is embedded into the first vocabulary.

3. The method for converting Excel files into complex structure table images as claimed in claim 2, characterized in that: Also includes: Performing a first decoding on the information embedded in the first vocabulary to obtain a feature tensor; The feature tensor is mapped to the vocabulary space through a linear layer to generate the first sequence of outputs.

4. The method for converting Excel files into complex structure table images as claimed in claim 1 or 3, characterized in that: Inputting the first feature information into the first branch includes: Weighting and integrating the key domain information of the table to obtain characteristic data of the key domain information; The feature data of key domain information is early fused and image dimensions are matched.

5. The method for converting Excel files to complex structure table images as claimed in claim 4, characterized in that: Also includes: Use the convolution kernel to sample the feature data, and use the set convolution kernel output and the segmentation map; The positioning information is obtained according to the segmentation map, the positioning information is input into the first recognition tool, and the output text is stored.

6. The method for converting Excel files into complex structure table images as claimed in claim 3, characterized in that: The first decoding includes: The target sequence is embedded by position and fused with the output features of the first encoder to obtain the input of the first decoder; The fused feature and mask tensors are input into the first decoder to obtain the feature tensor.

7. The method for converting Excel files to complex structure table images as claimed in claim 2, characterized in that: The first vocabulary embedding includes: Perform word embedding operation on each word in the target vocabulary, and perform position embedding operation on the sequence reference after word embedding; For each position and dimension, if the dimension is even, the value of the position embedding is calculated using the sine function; If the dimension is an odd number, the cosine function is used to calculate the value of the position embedding; Add the word embedding and the position embedding to get the information after the first vocabulary embedding.

8. A system using the method for converting Excel files oriented to complex structure table images as described in any one of claims 1 to 7, characterized in that: include: A feature extraction module acquires image data, and performs feature extraction on the image data using a first backbone network to obtain feature information; a transformation module, configured to perform a first transformation on the feature information to obtain first feature information, and input the first feature information into a first branch to obtain positioning information; An input module, configured to input the first feature information into a second branch based on the positioning information to obtain a first sequence; a replacement module, configured to replace the first sequence according to the positioning information and the first sequence to obtain a first code; The conversion module is used to convert the first code using a first tool to obtain an Excel file.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the method for converting Excel files for complex structure table images as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for converting an Excel file for complex structure table images as described in any one of claims 1 to 7.

Images