Instrument spreadsheet layout classification method, device and storage medium based on ResNet

Abstract

The present invention belongs to the technical field of electronic spreadsheet data processing, and in particular relates to a ResNet-based instrument electronic spreadsheet layout classification method. The method comprises first obtaining a readable original instrument electronic spreadsheet; modeling the structural information and cell content information of the readable original instrument electronic spreadsheet to obtain a structural tensor and a cell content tensor; concatenating the structural tensor and the cell content tensor to obtain a feature tensor of the instrument electronic spreadsheet; inputting the result into an improved ResNet-based table layout classification model for training, and saving the optimal table layout classification model; and inputting the instrument electronic spreadsheet to be layout-classified into the optimal table layout classification model, outputting the layout classification result, and comparing the result against a preset evaluation index, thereby outputting the comparison result. The present invention can solve the problem of poor flexibility of algorithms used in the prior art for processing electronic spreadsheet structural layout.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic spreadsheet data processing, and in particular relates to a ResNet-based instrument electronic spreadsheet layout classification method, device, and storage medium. Background Art

[0002] Spreadsheets are precise, flexible, easily shareable, and visual tools that provide fast, convenient, and real-time data entry and analysis. As a result, they are widely used in fields such as healthcare, finance, education, science, and engineering. In real-world business, numerous companies utilize spreadsheets for data analysis or as information storage. If companies fail to create well-designed spreadsheets or analyze incomplete data within them, they can suffer losses in profits or market share. The instrumentation industry is no exception, with numerous users using spreadsheets to submit their instrumentation inquiries. Often, instrumentation companies manually extract information from these spreadsheets. However, when dealing with large amounts of data and poorly readable spreadsheets, manual information extraction becomes time-consuming and error-prone, leading to inaccurate analysis and potentially business losses for instrumentation companies. Therefore, automated information extraction from instrumentation spreadsheets is crucial.

[0003] Since spreadsheets were originally designed for human use, they are easy for humans to process and understand. However, for machines, spreadsheets are not so easy to understand and process. For example, for the same data, the layout of the table can appear in many different forms depending on the tabulation habits of the tabulator. Humans can easily understand different spreadsheet layouts, but it is quite difficult for machines to do the same. Moreover, as semi-structured data, spreadsheets contain certain structural information. Directly converting spreadsheets to text will lose this structural information, so spreadsheets cannot be directly converted to text for processing. Therefore, the prerequisite for automated extraction of instrument spreadsheet information is to identify the layout structure of the instrument spreadsheet.

[0004] Currently, there is limited research on structure recognition for instrument spreadsheets and publicly available datasets of Chinese instrument spreadsheets. Most approaches to addressing the diverse structure and layout of spreadsheets rely on writing a large number of rule-based functions. While these algorithms can achieve certain processing results, they lack flexibility and require significant time to maintain and modify the written rule-based functions. Summary of the Invention

[0005] The technical problem solved by the present invention is to provide a method, device and storage medium for classifying the layout of instrument spreadsheets based on ResNet, so as to solve the problem of poor flexibility of the algorithms used in the prior art for processing the structural layout of spreadsheets.

[0006] The basic solution provided by the present invention: A method for classifying the layout of instrument spreadsheets based on ResNet, including:

[0007] S1: Obtain the original instrument spreadsheet, and screen out the readable original instrument spreadsheets from the obtained original instrument spreadsheets;

[0008] S2: Model the structural information of the readable original instrument spreadsheet and convert it into a structure tensor;

[0009] S3: Model the cell content information of the readable original instrument spreadsheet and convert it into a cell content tensor;

[0010] S4: Concatenate the structure tensor and the cell content tensor to obtain the feature tensor of the instrument spreadsheet;

[0011] S5: Build an improved table layout classification model based on ResNet, input the feature tensor of the instrument spreadsheet into the table layout classification model for training, and save the optimal table layout classification model;

[0012] S6: Input the instrument spreadsheet to be layout-classified into the optimal table layout classification model, output the layout classification result, and compare it with the preset evaluation index, and output the comparison result.

[0013] Further, the S1 includes:

[0014] Characterize the instrument spreadsheet as a two-dimensional tensor, and use C

[0018] ,

[0017] ,

[0016] , ,

[0019] , , , , To represent the value of the cell in the i(0 < i < n) row and j(0 < j < m) column of the instrument spreadsheet, where n is the maximum number of rows in the instrument spreadsheet S, and m is the maximum number of columns in the instrument spreadsheet S; the expression is:

[0015]

[0016] Among them, S represents the instrument spreadsheet.

[0017] Further, the expression of the structure tensor in S2 is: <000**********]]

[0018] F_struct = [Feature1, Feature2, Feature3, Feature4, Feature5]

[0019] Among them, the value of each cell in Feature1 is:​

[0020]

[0021] Among them, L ij Indicates the length of the string in the cell of row i (i≥0) and column j (j≥0);

[0022] The values ​​of each cell in Feature2 are:

[0023]

[0024] Among them, L mean represents the average length of the strings in all cells of the original instrument spreadsheet dataset, L s represents the standard deviation of the lengths of the strings in all cells of the original instrument spreadsheet dataset;

[0025] The values ​​of each cell in Feature3 are:

[0026]

[0027] Among them, M ij Represents the cell in row i (i ≥ 0) and column j (j ≥ 0);

[0028] The values ​​of each cell in Feature4 are:

[0029] C ij =i / m

[0030] Where m represents the maximum number of rows in the instrument spreadsheet;

[0031] The values ​​of each cell in Feature5 are:

[0032] C ij =j / n

[0033] Where n is the maximum number of columns in the instrument spreadsheet.

[0034] Furthermore, the cell content tensor in S3 is represented as:

[0035] F_value=[Feature6,Feature7,Feature8,Feature9,Feature10,Feature11,Feature12]

[0036] The values ​​of each cell in Feature6 are:

[0037]

[0038] Among them, V ijRepresents the value of the cell in row i (i≥0) and column j (j≥0), wherein the attribute noun library is established by collecting information from the instrument spreadsheet;

[0039] The values ​​of each cell in Feature7 are:

[0040]

[0041] The attribute value word library is established by collecting information from the instrument electronic form;

[0042] The values ​​of each cell in Feature8 are:

[0043]

[0044] The values ​​of each cell in Feature9 are:

[0045]

[0046] The values ​​of each cell in Feature10 are:

[0047]

[0048] The values ​​of each cell in Feature11 are:

[0049]

[0050] The values ​​of each cell in Feature12 are:

[0051]

[0052] Furthermore, the feature tensor expression in S4 is:

[0053] Feature_s=[F_struct,F_value]

[0054] Among them, F_struct and F_value are both three-dimensional tensors arranged in the order of FxRxC, where F represents the number of features in the instrument spreadsheet, R represents the number of rows in the instrument spreadsheet, and C represents the number of columns in the instrument spreadsheet.

[0055] Furthermore, the improved ResNet-based table layout classification model in S5 is specifically: the 7x7 convolution of the ResNet model is changed to a combination of 1x1 convolution and 3x3 convolution.

[0056] Furthermore, in S5, the table layout classification model is trained by using the focus loss, and the focus loss is expressed as:

[0057]

[0058] Among them, a t and γ represent the hyperparameters in focal loss, p t Represents the probability value of the result category predicted by the model.

[0059] An electronic device includes a processor and a memory, wherein the memory stores programs or instructions, and the processor executes the above-mentioned ResNet-based instrument spreadsheet layout classification method by calling the program or instructions stored in the memory.

[0060] A computer-readable storage medium stores a program or instruction, which enables a computer to execute the ResNet-based instrument spreadsheet layout classification method as described above.

[0061] The principles and advantages of this invention lie in modeling the original instrument spreadsheet's structural information and cell content information to obtain an instrument spreadsheet structure tensor and a cell content tensor, which are then concatenated to obtain an instrument spreadsheet feature tensor. This modeling approach fully extracts the information contained in the instrument spreadsheet. The obtained instrument spreadsheet feature tensor is then input into a ResNet-based spreadsheet layout classification algorithm for classification. This approach offers excellent flexibility and eliminates the need for time-consuming algorithm modification and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flowchart of an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of the structure features of the instrument electronic table and the content features of the instrument electronic table cells according to an embodiment of the present invention;

[0064] Figure 3 This is a confusion matrix diagram of an embodiment of the present invention using only the structural feature F_struct;

[0065] Figure 4 This is a confusion matrix diagram using only the cell content feature F_value in an embodiment of the present invention;

[0066] Figure 5 This is a confusion matrix diagram of Feature_s in an embodiment of the present invention;

[0067] Figure 6 This is a network structure diagram of ResNet18-Spreadsheet in an embodiment of the present invention;

[0068] Figure 7This is the confusion matrix diagram of using ResNet18 and ResNet18 - Spreadsheet in the embodiments of the present invention;

[0069] Figure 8 This is the confusion matrix diagram of using ResNet34 and ResNet34 - Spreadsheet in the embodiments of the present invention;

[0070] Figure 9 This is the confusion matrix diagram of using ResNet50 and ResNet50 - Spreadsheet in the embodiments of the present invention;

[0071] Figure 10 This is the confusion matrix diagram of using ResNet101 and ResNet101 - Spreadsheet in the embodiments of the present invention;

[0072] Figure 11 This is the schematic diagram of the electronic device in the embodiments of the present invention. Detailed implementation manners

[0073] The following is a further detailed description through specific implementation manners:

[0074] The marks in the attached drawings of the specification include: electronic device 400, processor 401, memory 402, input device 403, output device 404.

[0075] [[ID=​​​​​​​​​​​​​​​​​​​​​In this embodiment, the structure tensor obtained by modeling the structural information of the readable original instrument spreadsheet is a three-dimensional tensor F_struct. The three-dimensional tensor F_struct is composed of five two-dimensional tensors and can be expressed as:

[0082] F_struct=[Feature1,Feature2,Feature3,Feature4,Feature5]

[0083] Among them, the values ​​of each cell of Feature1 are:

[0084]

[0085] Among them, L ij Indicates the length of the string in the cell of row i (i≥0) and column j (j≥0);

[0086] The values ​​of each cell in Feature2 are:

[0087]

[0088] Among them, L mean represents the average length of the strings in all cells of the original instrument spreadsheet dataset, L s represents the standard deviation of the lengths of the strings in all cells of the original instrument spreadsheet dataset;

[0089] The values ​​of each cell in Feature3 are:

[0090]

[0091] Among them, M ij Represents the cell in row i (i ≥ 0) and column j (j ≥ 0);

[0092] The values ​​of each cell in Feature4 are:

[0093] C ij =i / m

[0094] Where m represents the maximum number of rows in the instrument spreadsheet;

[0095] The values ​​of each cell in Feature5 are:

[0096] C ij =j / n

[0097] Where n is the maximum number of columns in the instrument spreadsheet.

[0098] S3: Model the cell content information of the readable original instrument spreadsheet and convert it into a cell content tensor;

[0099] In this embodiment, the cell content tensor obtained by modeling the cell content information of the readable original instrument spreadsheet is a three-dimensional tensor, and the expression is:

[0100] F_value=[Feature6,Feature7,Feature8,Feature9,Feature10,Feature11,Feature12]

[0101] The values ​​of each cell in Feature6 are:

[0102]

[0103] Among them, V ij Represents the value of the cell in row i (i ≥ 0) and column j (j ≥ 0). The attribute noun library is established by collecting information from the instrument spreadsheet, such as "temperature", "caliber size", "pressure", "flow range", "medium", "main name", "form", etc.

[0104] The values ​​of each cell in Feature7 are:

[0105]

[0106] The attribute value word library is built by collecting information from the instrument spreadsheet; for example, "water", "titanium", "normal temperature", "normal pressure", "standard output", "integrated", etc.

[0107] The values ​​of each cell in Feature8 are:

[0108]

[0109] The values ​​of each cell in Feature9 are:

[0110]

[0111] The values ​​of each cell in Feature10 are:

[0112]

[0113] The values ​​of each cell in Feature11 are:

[0114]

[0115] The values ​​of each cell in Feature12 are:

[0116]

[0117] S4: Concatenate the structure tensor and the cell content tensor to obtain the feature tensor of the instrument spreadsheet;

[0118] In this embodiment, the feature tensor is represented as Feature_s, and its expression is:

[0119] Feature_s=[F_struct,F_value]

[0120] like Figure 2 As shown in the figure, the splicing process is as follows: first obtain F_struct and F_value, where F_struct and F_value are three-dimensional tensors arranged in the order of FxRxC (F represents the number of features of the instrument spreadsheet, R represents the number of rows of the instrument spreadsheet, and C represents the number of columns of the instrument spreadsheet). Then, F_struct and F_value are spliced ​​according to the dimension of the number of features of the instrument spreadsheet to obtain the feature tensor Feature_s of the instrument spreadsheet.

[0121] In order to verify the effect of the spliced ​​feature tensor Feature_s, as shown in Table 1 below, a set of ablation experiments are designed to analyze the performance improvement of the feature tensor Feature_s. The confusion matrix of F_struct, F_value, and Feature_s is as follows: Figure 3 、 Figure 4 and Figure 5 As shown in the figure, the effect of this experiment is that the spliced ​​Feature_s improves by 2.67%, 3.35%, 3.13% and 3.55% compared with F_struct in terms of Accuracy, Precision, Recall and Macro_F1; Feature_s improves by 16%, 23.63%, 23.28% and 24.8% compared with F_value in terms of Accuracy, Precision, Recall and Macro_F1.

[0122] Table 1 Ablation study results of different instrument spreadsheet feature tensors

[0123]

[0124] S5: Build an improved ResNet-based table layout classification model, input the feature tensor of the instrument spreadsheet into the table layout classification model for training, and save the optimal table layout classification model;

[0125] In this application, considering the comparison of model parameters and performance, ResNet is used as the initial model for modification. The 7x7 convolution of the ResNet model is changed to a combination of 1x1 convolution and 3x3 convolution, which is recorded as the ResNet-Spreadsheet network model. The network model structure is as follows: Figure 6 As shown, at the same time, the table layout classification model is trained. In this embodiment, the focus loss is used for training to adjust the weight of each category in the loss function during training. The expression is:

[0126]

[0127] Among them, a t and γ represent the hyperparameters in focal loss, p t Represents the probability value of the result category predicted by the model.

[0128] In this application, γ is set to 1, a t Set to [0.2, 0.26, 0.35, 0.2, 0.35, 0.35, 0.2, 0.45, 0.35].

[0129] At the same time, a set of comparative experiments are designed to analyze the effectiveness of the improved method by analyzing ResNet networks of different depths and the improved ResNet-Spreadsheet network, as shown in Table 2 below:

[0130] Table 2 Comparative study results between ResNet and ResNet-Spreadsheet

[0131]

[0132] In Table 2, the effects are shown as follows:

[0133] 1. ResNet18-Spreadsheet improves Accuracy, Precision, Recall, and Macro_F1 by 8%, 12.6%, 16.84%, and 16.37% compared to ResNet18;

[0134] 2. ResNet34-Spreadsheet improves Accuracy, Precision, Recall, and Macro_F1 by 6.67%, 6.18%, 13.04%, and 11.66% compared to ResNet34;

[0135] 3. ResNet50-Spreadsheet improves Accuracy, Precision, Recall, and Macro_F1 by 7.56%, 2.02%, 14.43%, and 10.74% respectively compared to ResNet50.

[0136] 4. ResNet101-Spreadsheet improves Accuracy, Precision, Recall, and Macro_F1 by 9.34%, 14.3%, 16.57%, and 14.34% respectively compared to ResNet101.

[0137] According to the experimental results, the combination of 1×1 convolution and 3×3 convolution to replace the 7×7 convolution of the first layer of the ResNet network has achieved significant improvement compared with the original ResNet network. Figure 7 、 Figure 8 、 Figure 9 、 Figure 10 They are the confusion matrix of ResNet18 and ResNet18-Spreadsheet, the confusion matrix of ResNet34 and ResNet34-Spreadsheet, the confusion matrix of ResNet50 and ResNet50-Spreadsheet, and the confusion matrix of ResNet101 and ResNet101-Spreadsheet.

[0138] S6: Input the electronic spreadsheet of the instruments to be classified into the table layout classification optimal model, output the layout classification result, compare it with the preset evaluation index, and output the comparison result.

[0139] The preset evaluation indicators include Accuracy, Precision, Recall, and F1 Score. The calculation method of Precision, Recall, and F1 Score is as follows:

[0140] Precision = TP / (TP+FP)

[0141] Recall = TP / (TP+FN)

[0142] F1=(2*Precision*Recall) / (Precision+Recall)

[0143] Among them, TP is the positive sample predicted by the model as the positive class, FP is the negative sample predicted by the model as the positive class, and FN is the positive sample predicted by the model as the negative class.

[0144] In this embodiment, the Accuracy is 93.78%, the Precision is 91.04%, the Recall is 89.64%, and the F1Score is 90.01%.

[0145] In another embodiment of this embodiment, an electronic device is also included, such as Figure 11 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0146] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0147] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the ResNet-based instrument spreadsheet layout classification method and / or other desired functions of any embodiment of the present invention described above. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0148] In one example, electronic device 400 may further include an input device 403 and an output device 404, which are interconnected via a bus system and / or other connection mechanisms (not shown). Input device 403 may include, for example, a keyboard, a mouse, etc. Output device 404 may output various information to the outside, including warning information, braking force, etc. Output device 404 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0149] Of course, to simplify, Figure 11 Only some of the components related to the present invention in the electronic device 400 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 400 may further include any other appropriate components according to specific application scenarios.

[0150] In addition to the above methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the ResNet-based instrument spreadsheet layout classification method provided by any embodiment of the present invention.

[0151] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0152] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to perform the steps of the ResNet-based instrument spreadsheet layout classification method provided in any embodiment of the present invention.

[0153] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0154] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme are not described in detail here. Ordinary technicians in the field are aware of all common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A ResNet-based instrument spreadsheet layout classification method, characterized by: include: S1: obtaining an original instrument spreadsheet, and screening out readable original instrument spreadsheets from the obtained original instrument spreadsheets; S2: Model the structural information of the readable original instrument spreadsheet and convert it into a structural tensor; S3: Model the cell content information of the readable original instrument spreadsheet and convert it into a cell content tensor; The cell content tensor in S3 is represented as: The values ​​of each cell are: in, express OK The value of the column cell, the attribute noun library and the following The attribute value vocabulary of the description is built by collecting information from the instrument spreadsheet; is the maximum number of rows in the instrument spreadsheet, is the maximum number of columns in the instrument spreadsheet; ， ; The values ​​of each cell are: The values ​​of each cell are: The values ​​of each cell are: The values ​​of each cell are: The values ​​of each cell are: The values ​​of each cell are: S4: Concatenate the structure tensor and the cell content tensor to obtain the feature tensor of the instrument spreadsheet; S5: Build an improved ResNet-based table layout classification model, input the feature tensor of the instrument spreadsheet into the table layout classification model for training, and save the optimal table layout classification model; S6: Input the electronic spreadsheet of the instruments to be classified into the table layout classification optimal model, output the layout classification result, compare it with the preset evaluation index, and output the comparison result.

2. The ResNet-based instrument spreadsheet layout classification method according to claim 1, characterized in that: Said S1 includes: Represent the instrument spreadsheet as a two-dimensional tensor and pass Indicates the instrument spreadsheet OK The value of the column cell, where ， ; where the expression is: in, Represents an instrument spreadsheet.

3. The ResNet-based instrument spreadsheet layout classification method according to claim 2, characterized in that: The structure tensor expression in S2 is: in, The values ​​of each cell are: in, express OK The length of the string in the column cell; where ， ; The values ​​of each cell are: in, represents the average length of the strings in all cells of the original instrument spreadsheet dataset, represents the standard deviation of the lengths of the strings in all cells of the original instrument spreadsheet dataset; The values ​​of each cell are: in, express OK column cells; where ， ; The values ​​of each cell are: in, Indicates the maximum number of rows in the instrument spreadsheet; The values ​​of each cell are: in, Indicates the maximum number of columns in the instrument spreadsheet.

4. The ResNet-based instrument spreadsheet layout classification method according to claim 3, characterized in that: The feature tensor expression in S4 is: in, and They are all three-dimensional tensors arranged as FxRxC, where F represents the number of features in the instrument spreadsheet, R represents the number of rows in the instrument spreadsheet, and C represents the number of columns in the instrument spreadsheet.

5. The ResNet-based instrument spreadsheet layout classification method according to claim 4, characterized in that: The improved ResNet-based table layout classification model in S5 is specifically as follows: the 7x7 convolution of the ResNet model is changed to a combination of 1x1 convolution and 3x3 convolution.

6. The ResNet-based instrument spreadsheet layout classification method according to claim 5, characterized in that: In S5, the table layout classification model is trained by using the focus loss, and the focus loss is expressed as: in, and represents the hyperparameter in focal loss, Represents the probability value of the result category predicted by the model.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction, and the processor executes the ResNet-based instrument spreadsheet layout classification method according to any one of claims 1 to 6 by calling the program or instruction stored in the memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the ResNet-based instrument spreadsheet layout classification method according to any one of claims 1 to 6.

Images