Table data labeling method and electronic device

Abstract

The application provides a table data labeling method and an electronic device. The method comprises the following steps: generating a first HTML label according to a first model and a recognition result of a target table, generating a second HTML label according to a second model and the recognition result of the target table, and generating a third HTML label according to a third model and the recognition result of the target table; performing difference comparison on the first HTML label and the second HTML label, determining first comparison information, and updating the second HTML label; performing difference comparison on the first HTML label and the third HTML label, determining second comparison information, and updating the third HTML label; performing election and label updating according to the first HTML label, the updated second HTML label and the updated third HTML label, obtaining a target HTML label, and obtaining a labeling result of the target table according to the target HTML label. The accuracy of labeling can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method for tabular data annotation and an electronic device. Background Technology

[0002] Fields like finance involve numerous tables, necessitating accurate table recognition. With the rapid development of Artificial Intelligence (AI) technology, AI models can now perform table recognition on images. However, before training, these AI models require table annotation to obtain training data.

[0003] In existing technologies, automated table annotation is mainly achieved through traditional small models or end-to-end large models. Traditional small models focus on aligning physical coordinates with local features, and table annotation primarily includes labeling cell bounding rectangles, row and column separators, and text lines from Optical Character Recognition (OCR). End-to-end large models can be multimodal models such as Vision-Language Models (VLMs). These multimodal models directly map images into structured sequences, recognize the topological logic of tables, and perform table annotation accordingly.

[0004] However, traditional small models struggle to automatically identify complex semantic errors, while multimodal large models, due to their error-correcting capabilities or even their ability to detect errors, may automatically complete missing text in an image or discard some text. Therefore, both traditional small models and multimodal large models have certain limitations. Summary of the Invention

[0005] The purpose of this application is to provide a tabular data annotation method and electronic device to address the shortcomings of the prior art, thereby solving the problem that both traditional small models and multimodal large models in the prior art have certain limitations.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: Firstly, this application provides a method for labeling tabular data, including: Based on the recognition result of the first model for the target table, generate the first HTML tag corresponding to the first model; based on the recognition result of the second model for the target table, generate the second HTML tag corresponding to the second model; based on the recognition result of the third model for the target table, generate the third HTML tag corresponding to the third model. The difference between the first HTML tag and the second HTML tag is compared to determine the first comparison information of the second HTML tag relative to the first HTML tag, and the second HTML tag is updated according to the first comparison information to obtain the updated second HTML tag; The first HTML tag and the third HTML tag are compared to determine the second comparison information of the third HTML tag relative to the first HTML tag, and the third HTML tag is updated according to the second comparison information to obtain the updated third HTML tag; Based on the first HTML tag, the updated second HTML tag, and the updated second HTML tag, an election and tag update are performed to obtain the target HTML tag, and the annotation result of the target table is obtained based on the target HTML tag.

[0007] Secondly, this application provides a tabular data annotation device, comprising: The generation module is used to generate a first hypertext markup language (HTML) tag corresponding to the first model based on the recognition result of the first model for the target table, generate a second HTML tag corresponding to the second model based on the recognition result of the second model for the target table, and generate a third HTML tag corresponding to the third model based on the recognition result of the third model for the target table. The first comparison module is used to perform a difference comparison between the first HTML tag and the second HTML tag, determine the first comparison information of the second HTML tag relative to the first HTML tag, and update the second HTML tag according to the first comparison information to obtain the updated second HTML tag. The second comparison module is used to perform a difference comparison between the first HTML tag and the third HTML tag, determine the second comparison information of the third HTML tag relative to the first HTML tag, and update the third HTML tag according to the second comparison information to obtain the updated third HTML tag; The annotation module is used to perform election and tag updating based on the first HTML tag, the updated second HTML tag, and the updated second HTML tag to obtain the target HTML tag, and to obtain the annotation result of the target table based on the target HTML tag.

[0008] Thirdly, this application provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to perform the steps of the tabular data annotation method as described in the first aspect above.

[0009] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the tabular data annotation method described in the first aspect.

[0010] Fifthly, this application provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the tabular data annotation method described in the first aspect.

[0011] The table data annotation method and electronic device provided in this application first generate a first HTML tag for the target table using a first model, a second HTML tag for the target table using a second model, and a third HTML tag for the target table using a third model. Then, it determines first comparison information of the second HTML tag relative to the first HTML tag and second comparison information of the third HTML tag relative to the first HTML tag. Since the first model accurately identifies the cell coordinates of the table, using the first model as a benchmark for difference comparison ensures the reliability and accuracy of the difference comparison results, as well as the subsequent election and update results. Based on the first and second comparison information, updated second and third HTML tags can be obtained, ensuring that the updated second and third HTML tags record difference information relative to the first HTML tag. Then, based on the first HTML tag, the updated second HTML tag, and the updated third HTML tag, election and tag updating are performed to obtain the target HTML tag. Since the updated second and third HTML tags record differences from the first HTML tag, based on these differences and the election operation, the most accurate content can be determined from the recognition results of the three models, thus significantly improving the accuracy of the resulting table annotations compared to the annotations obtained through a single model. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a scenario example diagram illustrating the tabular data annotation method of this application; Figure 2 A flowchart illustrating the tabular data annotation method provided in this application; Figure 3 A schematic diagram illustrating the difference comparison process of the tabular data annotation methods provided in this application; Figure 4 A schematic diagram of the election update process for the tabular data annotation method provided in this application; Figure 5 A schematic diagram illustrating the process of determining target HTML tags for the tabular data annotation method provided in this application; Figure 6 A modular structure diagram of the tabular data annotation device provided in this application; Figure 7 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0015] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0016] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0017] Traditional small-scale models are highly accurate in recognizing the physical coordinates of tables, but they struggle to automatically identify complex semantic errors. Multimodal large-scale models, due to their error-correction capabilities or "illusion" mechanisms, may automatically complete missing text in the image or discard some text, potentially leading to discrepancies between the recognition results and the actual tables. Therefore, both traditional small-scale models and multimodal large-scale models have limitations; using only one cannot yield accurate table annotation results.

[0018] Based on the above problems, this application proposes a table data annotation method. Based on the table recognition results of the traditional small model and two multimodal large models, the method compares the differences between the two multimodal large models and the traditional small model, and elects and updates the recognition results of the three models based on the difference comparison results, thereby obtaining accurate table annotation results.

[0019] This application can be applied to scenarios where AI models are required for table recognition. Figure 1 This is a scenario example diagram illustrating the tabular data annotation method of this application, such as... Figure 1 As shown, in the financial field, it is necessary to train an AI model with accurate text recognition and table recognition capabilities to help users extract and analyze financial-related document content. To train this AI model, a large amount of labeled table data is needed as training samples. Therefore, based on the method provided in this application, a large amount of labeled table data can be automatically generated, and this labeled data can be used to train the AI ​​model, enabling it to have strong table recognition capabilities.

[0020] Figure 2 This is a flowchart illustrating the tabular data annotation method provided in this application. The subject executing this method can be any electronic device with computing capabilities. Figure 2 As shown, the method may include: S201. Generate the first HyperText Markup Language (HTML) tag corresponding to the first model based on the recognition result of the first model for the target table, generate the second HTML tag corresponding to the second model based on the recognition result of the second model for the target table, and generate the third HTML tag corresponding to the third model based on the recognition result of the third model for the target table.

[0021] Optionally, the target table can be, for example, a table in a file or image. By inputting the file or image containing the target table into the first model, the second model, and the third model respectively, the recognition results of the first model, the second model, and the third model for the target table can be obtained.

[0022] Optionally, the first model mentioned above is a traditional small model, which can be a detection model based on TATR (TableTransformer), such as the RT-DETR series models. The second and third models mentioned above are multimodal large models, such as Mineru 2.5, Paddleocr, GLM-OCR, etc.

[0023] Optionally, since the recognition result of the first model for the target table is not in HTML format, it can be converted to obtain the aforementioned first HTML tag, thereby facilitating subsequent comparison analysis and election update processes. Correspondingly, the recognition result of the first model for the target table can also be converted to obtain the second and third HTML tags.

[0024] S202. Perform a difference comparison between the first HTML tag and the second HTML tag to determine the first comparison information of the second HTML tag relative to the first HTML tag, and update the second HTML tag according to the first comparison information to obtain the updated second HTML tag.

[0025] The updated second HTML tag contains information about the differences between the second HTML tag and the first HTML tag.

[0026] Optionally, when comparing the differences between the first HTML tag and the second HTML tag, the first HTML tag is used as the benchmark to identify the differences between the second HTML tag and the first HTML tag. Since the first model is a traditional small model, which has high accuracy in recognizing the physical coordinates of the table, it can provide precise cell space coordinates. These precise cell space coordinates can serve as the spatial benchmark for difference comparison and cell positioning. Therefore, using the first HTML tag corresponding to the first model as the benchmark for difference comparison ensures the reliability and accuracy of the difference comparison results, as well as subsequent election and update results.

[0027] Optionally, after identifying the first comparison information of the second HTML tag relative to the first HTML tag, the second HTML tag is updated according to the first comparison information, so that the updated second HTML tag records the difference between the second HTML tag and the first HTML tag, and this difference is used for election and updating of HTML tags in subsequent processes.

[0028] S203. Perform a difference comparison between the first HTML tag and the third HTML tag to determine the second comparison information of the third HTML tag relative to the first HTML tag, and update the third HTML tag according to the second comparison information to obtain the updated third HTML tag.

[0029] The updated third HTML tag contains information about the differences between the third HTML tag and the first HTML tag.

[0030] Optionally, when comparing the differences between the first HTML tag and the third HTML tag, the first HTML tag is used as the benchmark to identify the differences between the third HTML tag and the first HTML tag. Since the first model is a traditional small model, which has high accuracy in recognizing the physical coordinates of the table, it can provide precise cell space coordinates. These precise cell space coordinates can serve as the spatial benchmark for difference comparison and cell positioning. Therefore, using the first HTML tag corresponding to the first model as the benchmark for difference comparison ensures the reliability and accuracy of the difference comparison results, as well as subsequent election and update results.

[0031] Optionally, after identifying the second comparison information of the third HTML tag relative to the first HTML tag, the second HTML tag is updated according to the second comparison information, so that the updated third HTML tag records the difference between the third HTML tag and the first HTML tag, and this difference is used for election and HTML tag updating in subsequent processes.

[0032] S204. Based on the first HTML tag, the updated second HTML tag, and the updated second HTML tag, perform election and tag update to obtain the target HTML tag, and obtain the annotation result of the target table based on the target HTML tag.

[0033] Optionally, an election involving three parties is conducted based on the content of the same cell in the first HTML tag, the updated second HTML tag, and the updated second HTML tag. The three parties include the first HTML tag, the updated second HTML tag, and the updated second HTML tag. Through the election, the accurate content of the same cell can be determined. Then, based on the accurate content of the same cell, the first HTML tag, the updated second HTML tag, and the updated second HTML tag are updated accordingly. Finally, one of the three parties is selected as the target HTML tag based on the number of times each party is selected during the election process. The annotation result of the target table is then obtained based on the target HTML tag.

[0034] In this embodiment, firstly, a first HTML tag for the target table using a first model, a second HTML tag for the target table using a second model, and a third HTML tag for the target table using a third model are generated. Then, first comparison information of the second HTML tag relative to the first HTML tag and second comparison information of the third HTML tag relative to the first HTML tag are determined. Since the first model accurately identifies the cell coordinates of the table, using the first model as a benchmark for difference comparison ensures the reliability and accuracy of the difference comparison results, as well as subsequent election and update results. Based on the first and second comparison information, updated second and third HTML tags are obtained, ensuring that the updated second and third HTML tags record difference information relative to the first HTML tag. Then, based on the first HTML tag, the updated second HTML tag, and the updated third HTML tag, election and tag updating are performed to obtain the target HTML tag. Since the updated second and third HTML tags record difference information relative to the first HTML tag, based on this difference information and the election operation, the content with the highest accuracy can be determined from the recognition results of the three models, thus significantly improving the accuracy of the obtained table annotation results compared to the annotation results obtained through a single model.

[0035] As an optional implementation, the process of generating the first HTML tag corresponding to the first model based on the recognition result of the first model for the target table in step S201 above may include: The recognition results of the first model for the target table are standardized to obtain the first HTML tag. The first HTML tag includes at least: row identifier, row placeholder, column identifier, column placeholder, cell identifier, cell content, and cell coordinate information.

[0036] Optionally, the recognition result of the first model for the target table may be in the form of natural language description text, etc. Therefore, the recognition result can be standardized to obtain standard first HTML tags. The first HTML tags are used to record table information. Here, row identifiers can be row numbers, column identifiers can be column numbers, cell identifiers can be cell sequence numbers, row placeholders can be the number of atomic rows spanned by a row, and column placeholders can be the number of atomic columns spanned by a column. Cell content refers to the text within the cell, etc. Cell coordinate information can be the cell's position coordinates in the image. For example, during standardization, the recognition result can be parsed one by one to obtain the row number, row placeholder, column number, column placeholder, cell sequence number, cell content, and cell coordinates, and then corresponding HTML tag names can be added to this information. After parsing and adding tag names to all rows, the aforementioned first HTML tags can be obtained.

[0037] It is worth noting that HTML tags correspond to tables. Therefore, in the following embodiments, a row in an HTML tag specifically refers to a row in the table corresponding to the HTML tag.

[0038] Optionally, for the second and third models, the recognition results can also be standardized to obtain second and third HTML tags; the specific process will not be elaborated here. The second and third HTML tags may include the row identifier, row placeholder, column identifier, column placeholder, cell identifier, and cell content mentioned above. However, since the second and third models are less capable of recognizing cell spatial coordinates than the first model, the second and third HTML tags may not include the aforementioned cell coordinate information.

[0039] In this embodiment, by standardizing the recognition results of the first model, standard first HTML tags can be obtained, which facilitates subsequent comparison analysis and election update processes.

[0040] The process of generating the updated second HTML tag in step S202 is described below. The process of generating the updated third HTML tag in step S203 is the same as described below, except that the second HTML tag is replaced with the third HTML tag. Therefore, the specific execution process of S203 will not be described again.

[0041] Figure 3 A flowchart illustrating the difference comparison process for the tabular data annotation methods provided in this application is shown below. Figure 3 As described above, step S202 may include: S301. Perform row comparison, cell comparison, and character comparison on the first HTML tag and the second HTML tag to obtain first comparison information, which includes: row comparison result, cell comparison result, and character comparison result.

[0042] Optionally, when comparing the first HTML tag and the second HTML tag, the comparison can be performed line by line in order. When comparing a line, the comparison is performed in the hierarchical order of line, cell, and character. For example, for the current line of the second HTML tag, the current line is first compared with the corresponding line in the first HTML tag to obtain the line comparison result. Then, each cell in the current line is compared one by one. For example, when comparing the current cell, the current cell is compared with the corresponding cell in the corresponding line of the first HTML tag to obtain the cell comparison result. Then, each character in the current cell is compared with the corresponding character in the corresponding cell of the corresponding line of the first HTML tag to obtain the character comparison result.

[0043] S302. Based on the first comparison information above, set the highlight attribute for the target element in the second HTML tag to obtain the updated second HTML tag. The target element includes a row, cell, or character.

[0044] Optionally, the aforementioned first comparison result includes row comparison results, cell comparison results, and character comparison results. Highlight attributes can be set for target elements in the second HTML tag based on these comparison results. For example, if the row comparison result indicates that a row in the second HTML tag is redundant relative to the first HTML tag (i.e., the content of that row in the second HTML tag does not exist in the first HTML tag), then a specific highlight attribute can be set for that row in the second HTML tag. As another example, if the cell comparison result indicates that a cell in a row of the second HTML tag is modified relative to the first HTML tag (i.e., the content of that cell in the second HTML tag differs from the content of the corresponding cell in the first HTML tag), then a specific highlight attribute can be set for that cell in the second HTML tag.

[0045] In this embodiment, by comparing the first HTML tag and the second HTML tag at the row, cell, and character levels, the row comparison results, cell comparison results, and character comparison results can be obtained. Based on these comparison results, corresponding highlight attributes can be set in the second HTML tag. Because the highlight attributes are set, the difference information can be accurately extracted in the subsequent election, and an accurate election can be performed accordingly.

[0046] As an optional implementation, step S301 above may include: Determine the line alignment result of the i-th line in the second HTML tag relative to the first HTML tag. This line alignment result is used to indicate the editing status of the i-th line in the second HTML tag relative to the j-th line in the first HTML tag. The editing status includes: modified, redundant, or consistent.

[0047] If the line comparison result of the i-th row in the second HTML tag relative to the line comparison result of the first HTML tag is modified, and the line comparison result of the (i-1)-th row in the second HTML tag relative to the line comparison result of the first HTML tag is consistent, and the line comparison result of the (i+1)-th row in the second HTML tag relative to the line comparison result of the first HTML tag is consistent, then the cell comparison result of the p-th cell in the i-th row of the second HTML tag relative to the j-th row of the first HTML tag is determined. This cell comparison result is used to indicate the editing status of the p-th cell in the i-th row of the second HTML tag relative to the q-th cell in the j-th row of the first HTML tag.

[0048] If the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell in the j-th row of the first HTML tag is modified, then the character comparison result of the m-th character in the p-th cell in the i-th row of the second HTML tag with the q-th cell in the j-th row of the first HTML tag is determined. This character comparison result is used to indicate the editing status of the m-th character in the p-th cell in the i-th row of the second HTML tag with the n-th character in the q-th cell in the j-th row of the first HTML tag.

[0049] Optionally, the i-th row mentioned above can be any row in the second HTML tag, and the j-th row mentioned above refers to the row corresponding to the i-th row in the second HTML tag in the first HTML tag. When determining the row alignment result of the i-th row in the second HTML tag relative to the first HTML tag, the j-th row corresponding to the i-th row in the second HTML tag in the first HTML tag can be determined, i.e., the value of j can be determined. Based on this, the editing state of the i-th row relative to the j-th row can be determined. This editing state can be modified, redundant, or consistent. Modification means that the content of the i-th row is partially inconsistent with the content of the j-th row. Redundancy means that the content of the i-th row does not exist in the first HTML tag and belongs to a new row. It is worth noting that in this case, the value of j can be the row identifier of any row in the first HTML tag. Consistency means that the content of the i-th row is consistent with the content of the j-th row. As an example, the row alignment result of the i-th row in the second HTML tag relative to the first HTML tag can be determined using an edit distance algorithm, and the row alignment result can be represented by the following state function.

[0050] row_stat(b1_i->a)[0]=(stat, j) Where b1 represents the second HTML tag, b1_i represents the i-th line in the second HTML tag, a represents the first HTML tag, stat represents the editing state, and j represents the j-th line in a corresponding to the i-th line in b1.

[0051] Optionally, the aforementioned (i-1)th row refers to the row preceding the i-th row, and the aforementioned (i+1)th row refers to the row following the i-th row. If the row comparison result of the i-th row in the second HTML tag relative to the row comparison result of the first HTML tag is modified, and the row comparison result of the (i-1)th row in the second HTML tag relative to the row comparison result of the first HTML tag is consistent, and the row comparison result of the (i+1)th row in the second HTML tag relative to the row comparison result of the first HTML tag is consistent, then the cell comparison result corresponding to the p-th cell in the i-th row is determined. That is, when the above three conditions are met simultaneously, cell comparison is further performed. Specifically, if the row comparison result of the (i-1)th row in the second HTML tag is consistent with the row comparison result of the first HTML tag, and the row comparison result of the (i+1)th row in the second HTML tag is consistent with the row comparison result of the first HTML tag, it indicates that the rows before and after the i-th row are consistent, and the positioning of the i-th row is accurate. Therefore, the premise of the comparison of the i-th row can be guaranteed to be correct, thereby ensuring the reliability of the comparison. The p-th cell mentioned above can be any cell in the i-th row of the second HTML tag, and the q-th cell mentioned above refers to the cell in the j-th row of the first HTML tag corresponding to the p-th cell in the i-th row of the second HTML tag. When determining the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell in the j-th row of the first HTML tag, the cell in the i-th row of the second HTML tag corresponding to the cell in the j-th row of the first HTML tag can be determined, i.e., the value of q can be determined. Based on this, the editing state of the p-th cell in the i-th row relative to the q-th cell in the j-th row can be determined. This editing state can be modified, redundant, or consistent. Modified means that the content of the p-th cell is partially inconsistent with the content of the q-th cell. Redundant means that the content of the p-th cell does not exist in the j-th row of the first HTML tag and is a new cell. It is worth noting that in this case, the value of q can be the cell identifier of any cell in the j-th row of the first HTML tag. Consistent means that the content of the p-th cell is consistent with the content of the q-th cell. As an example, the cell comparison result of the i-th row p-th cell in the second HTML tag relative to the j-th row of the first HTML tag can be determined by the edit distance algorithm, and the cell comparison result can be represented by the following state function.

[0052] cell_stat(b1_i,p->a_j)[0]=(stat, q) Where b1_i represents the i-th row in the second HTML tag, b1_i,p represents the p-th cell in the i-th row of the second HTML tag, a_j represents the j-th row of the first HTML tag, stat represents the editing state, and q represents the cell in the j-th row of the corresponding element in a corresponding element in the i-th row of b1.

[0053] Optionally, if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell in the j-th row of the first HTML tag is modified, then the comparison result of the m-th character in the p-th cell of the i-th row of the second HTML tag with the character in the q-th cell of the j-th row of the first HTML tag is determined. That is, when the p-th cell is modified relative to its corresponding q-th cell, further comparison of characters within the cell is performed. The m-th character can be any character in the p-th cell of the i-th row of the second HTML tag, and the n-th character refers to the character in the q-th cell of the j-th row of the first HTML tag corresponding to the m-th character in the p-th cell of the i-th row of the second HTML tag. When determining the character comparison result of the m-th character in the p-th cell of the i-th row of the second HTML tag relative to the q-th cell of the j-th row of the first HTML tag, we can determine the cell corresponding to the m-th character in the p-th cell of the i-th row of the second HTML tag in the q-th cell of the j-th row of the first HTML tag, i.e., determine the value of n. Based on this, we determine the editing state of the m-th character in the p-th cell of the i-th row relative to the n-th character in the q-th cell of the j-th row. This editing state can be modified, redundant, or consistent. Modification means that the content of the m-th character is partially inconsistent with the content of the n-th character. Redundancy means that the content of the m-th character does not exist in the q-th cell of the first HTML tag and is a new character. It is worth noting that in this case, the value of n can be the character identifier of any character in the q-th cell of the j-th row of the first HTML tag. Consistency means that the content of the m-th character is consistent with the content of the n-th character. As an example, the character alignment result of the m-th character in the p-th cell of the i-th row of the second HTML tag relative to the q-th cell of the j-th row of the first HTML tag can be determined by the edit distance algorithm, and the character alignment result can be represented by the following state function.

[0054] char_stat(b1_i,p,m->a_j,q)=(stat, n) Where b1_i represents the i-th row in the second HTML tag, b1_i,p,m represents the m-th character in the p-th cell of the i-th row in the second HTML tag, a_j,q represents the q-th cell of the j-th row in the first HTML tag, stat represents the edit state, and n represents the character in the q-th cell of the j-th row in a corresponding to the m-th character in the p-th cell of the i-th row in b1.

[0055] In this embodiment, by performing row comparison, cell comparison, and character comparison layer by layer, the completeness and accuracy of the comparison can be guaranteed.

[0056] As an optional implementation, step S302 above may include: If the i-th line in the second HTML tag is redundant compared to the line of the first HTML tag, then the first highlight attribute is set for the i-th line in the second HTML tag.

[0057] If the comparison result of the i-th row in the second HTML tag with the row of the first HTML tag is modified, and the comparison result of the (i-1)-th row in the second HTML tag with the row of the first HTML tag is consistent, and the comparison result of the (i+1)-th row in the second HTML tag with the row of the first HTML tag is consistent; if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is redundant, then set the second highlight attribute for the p-th cell in the i-th row of the second HTML tag; if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is modified, then set the third highlight attribute for the p-th cell in the i-th row of the second HTML tag.

[0058] If the comparison result of the i-th row in the second HTML tag relative to the row of the first HTML tag is modified, and the comparison result of the (i-1)-th row in the second HTML tag relative to the row of the first HTML tag is consistent, and the comparison result of the (i+1)-th row in the second HTML tag relative to the row of the first HTML tag is consistent, and the comparison result of the p-th cell in the i-th row of the second HTML tag relative to the cell in the j-th row of the first HTML tag is modified; if the comparison result of the m-th character in the p-th cell of the i-th row of the second HTML tag relative to the character in the q-th cell of the j-th row of the first HTML tag is redundant, then set the fourth highlight attribute for the m-th character in the p-th cell of the i-th row of the second HTML tag; if the comparison result of the m-th character in the p-th cell of the i-th row of the second HTML tag relative to the character in the q-th cell of the j-th row of the first HTML tag is modified, then set the fifth highlight attribute for the m-th character in the p-th cell of the i-th row of the second HTML tag.

[0059] Optionally, if the i-th line in the second HTML tag is redundant compared to the line comparison of the first HTML tag, it indicates that the i-th line is a new line, and therefore a first highlight attribute is set for the i-th line in the second HTML tag. This first highlight attribute could be, for example, red highlighting. Specifically, CSS styles can be added to the i-th line in the first HTML tag to set the first highlight attribute for that line.

[0060] Optionally, if the row i in the second HTML tag is modified relative to the row comparison result of the first HTML tag, and the (i-1)th row in the second HTML tag is consistent with the row comparison result of the first HTML tag, and the (i+1)th row in the second HTML tag is consistent with the row comparison result of the first HTML tag, then further cell processing is performed. This can be divided into two cases. In the first case, if the row p in the second HTML tag is redundant relative to the row j in the first HTML tag, then a second highlight attribute is set for the row p in the second HTML tag. That is, if the row p is a new cell compared to the corresponding row q, then a second highlight attribute is set for the row p. This second highlight attribute can be, for example, red highlight. Specifically, CSS styles can be added to the row p in the second HTML tag to set the first highlight attribute for the row p. In the second case, if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell in the j-th row of the first HTML tag is modified, then a third highlight attribute is set for the p-th cell in the i-th row of the second HTML tag. That is, if there is a partial inconsistency between the p-th cell and its corresponding q-th cell, then a third highlight attribute is set for the p-th cell. This third highlight attribute could be, for example, yellow highlighting. Specifically, a CSS style can be added to the p-th cell in the i-th row of the second HTML tag to set the third highlight attribute for the p-th cell in the i-th row.

[0061] Optionally, if the comparison result of the i-th row in the second HTML tag with the row of the first HTML tag is modified, and the comparison result of the (i-1)-th row in the second HTML tag with the row of the first HTML tag is consistent, and the comparison result of the (i+1)-th row in the second HTML tag with the row of the first HTML tag is consistent, and the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is modified, then further processing of the cells in the i-th row is performed. This can be divided into two cases. In the first case, if the comparison result of the m-th character in the p-th cell of the i-th row of the second HTML tag with the character in the q-th cell of the j-th row of the first HTML tag is redundant, then a fourth highlight attribute is set for the m-th character in the p-th cell of the i-th row of the second HTML tag. That is, if the m-th character is a new character compared to the corresponding n-th character, then a fourth highlight attribute is set for the p-th cell. This fourth highlight attribute can be, for example, red highlight. Specifically, the font of the m-th character in the p-th cell of the i-th row of the second HTML tag can be set to red highlight. In the second case, if the m-th character in the p-th cell of the i-th row of the second HTML tag is modified compared to the character in the q-th cell of the j-th row of the first HTML tag, then the fifth highlight attribute is set for the m-th character in the p-th cell of the i-th row of the second HTML tag. That is, if the m-th character is inconsistent with its corresponding n-th character, then the fifth highlight attribute is set for the p-th cell. This fifth highlight attribute can be, for example, yellow highlighting. Specifically, the font of the m-th character in the p-th cell of the i-th row of the second HTML tag can be set to red highlighting.

[0062] In this embodiment, based on the actual comparison results, corresponding highlight attributes are added to elements at all levels in the HTML tags. This can be used in the subsequent election and manual annotation processes, improving the efficiency and accuracy of table annotation. For example, in the subsequent election update process, cells with highlight attributes are updated. In the subsequent manual annotation process, users manually confirm and update rows, cells, and characters with highlight attributes.

[0063] Figure 4 A schematic diagram of the election update process for the tabular data annotation method provided in this application is shown below. Figure 3 As described above, step S204 may include: S401. Based on the baseline evaluation results of the first model, the second model, and the third model, determine the priority of the first model, the second model, and the third model.

[0064] Optionally, a batch of tabular data can be provided in advance, and the correct results of this batch of data can be determined manually or automatically. This batch of tabular data is then input into the first model, the second model, and the third model respectively to obtain the recognition results of the first model, the second model, and the third model. Based on the correct results, the first model, the second model, and the third model are evaluated, for example, the character accuracy and structural accuracy of each model are evaluated. Based on the evaluation results, the score of each model is calculated, and the model with the highest score is assigned the highest priority, and so on, to obtain the respective priorities of the first model, the second model, and the third model.

[0065] S402. Based on the cells with the highlight attribute in the updated second HTML tag, determine the first set of cells corresponding to the updated second HTML tag, and determine the first set of mapped cells corresponding to the first HTML tag based on the first set of cells. Each cell in the first set of cells corresponds to a cell in the first set of mapped cells.

[0066] Optionally, the first cell set includes multiple cells, each of which is a cell with a highlight attribute in the updated second HTML tag. It's worth noting that cells with a highlight attribute include those with the second and third highlight attributes mentioned earlier. Furthermore, each cell in the first cell set corresponds to a cell in the first HTML tag, and this corresponding cell is used as a cell in the first mapped cell set. That is, the first mapped cell set includes multiple cells, each of which is a cell in the first HTML tag. And there is a one-to-one correspondence between the cells in the first cell set and the cells in the first mapped cell set.

[0067] S403. Based on the cells with the highlighted attribute in the updated third HTML tag, determine the second set of cells corresponding to the updated third HTML tag, and determine the second set of mapped cells corresponding to the first HTML tag based on the second set of cells. Each cell in the second set of cells corresponds to a cell in the second set of mapped cells.

[0068] Optionally, the second cell set includes multiple cells, each of which is a cell with a highlighted attribute in the updated third HTML tag. It's worth noting that cells with a highlighted attribute include those with both the second and third highlighted attributes mentioned earlier. Simultaneously, each cell in the second cell set corresponds to a cell in the first HTML tag, and this corresponding cell is used as a cell in the second mapped cell set. That is, the second mapped cell set includes multiple cells, each of which is a cell in the first HTML tag. Furthermore, there is a one-to-one correspondence between the cells in the second cell set and the cells in the second mapped cell set.

[0069] S404. Based on the above priority, the above first cell set, the above first mapping cell set, the above second cell set, and the above second mapping cell set, perform election and tag update to obtain the above target HTML tag.

[0070] Optionally, the above priorities indicate the order of priority among the first, second, and third models. Furthermore, the first and second cell sets include highlighted cells—cells where the recognition results differ between the models. Therefore, by jointly analyzing the first cell set with its corresponding first mapping cell set, and the second cell set with its corresponding second mapping cell set, a more reliable result can be determined. Based on this, updating each HTML tag with the more reliable result ensures the accuracy of the final target HTML tag.

[0071] The following describes the specific process of step S402. The execution process of step S403 is the same as this process, except that the cell name is replaced, so it will not be described again.

[0072] As an optional implementation, step S402 above may include: Add the cells with the highlight attribute in the updated second HTML tag to the first cell set; iterate through each cell in the first cell set, and for the first current cell, determine the corresponding cell in the first HTML tag based on the cell comparison result of the first current cell, add the corresponding cell to the first mapped cell set, and establish the mapping relationship between the first current cell and the corresponding cell.

[0073] Optionally, as mentioned above, when performing cell difference comparison, the cell in the second HTML tag corresponding to the cell in the first HTML tag can be determined based on the edit distance algorithm. Accordingly, in this embodiment, based on the aforementioned information, the cell corresponding to the first current cell in the first HTML tag can be obtained, and the corresponding cell can be added to the first mapped cell set. Simultaneously, the mapping relationship between the first current cell and its corresponding unit can be recorded; for example, a mapping table can be created to record the mapping relationship between the first current cell and its corresponding unit.

[0074] In this embodiment, based on the cell comparison result corresponding to the first current cell, the cell corresponding to the first current cell in the first HTML tag can be accurately determined, and thus an accurate first mapping cell set can be established.

[0075] Figure 5 A schematic diagram illustrating the process of determining target HTML tags for the tabular data annotation method provided in this application is shown below. Figure 5 As described above, step S404 may include: S501. Determine the intersection of the first mapping cell set and the second mapping cell set, the first difference of the first mapping cell set relative to the second mapping cell set, and the second difference of the second mapping cell set relative to the first mapping cell set.

[0076] Optionally, the intersection mentioned above refers to the set of cells that are simultaneously in both the first mapped cell set and the second mapped cell set. For example, a cell in the first HTML tag that is both a cell in the first mapped cell set and a cell in the second mapped cell set is a cell in the intersection.

[0077] Optionally, the first difference set refers to the set of cells that are only in the first mapped cell set and not in the second mapped cell set. The second difference set refers to the set of cells that are only in the second mapped cell set and not in the first mapped cell set.

[0078] S502. Based on the above intersection, the first cell set, and the second cell set, elect and update the labels of the cells in the intersection, and update the selection count of each model according to the update results.

[0079] As an optional implementation, the cells in the intersection are traversed. For the second current cell, the first comparison cell corresponding to the second current cell in the first cell set is determined, and the second comparison cell corresponding to the second current cell in the second cell set is determined. The cell contents of the second current cell, the first comparison cell, and the second comparison cell are compared to obtain the first best cell content. The first best cell content is updated in the first HTML tag, the updated second HTML tag, and the updated third HTML tag. The selection count of the model corresponding to the cell to which the first best cell content belongs is incremented by one.

[0080] Optionally, for the second current cell in the intersection, the first comparison cell corresponding to the second current cell in the first cell set and the second comparison cell corresponding to the second current cell in the second cell set can be found according to the mapping relationship recorded in the aforementioned embodiments. Based on this, the cell contents of the three cells are compared. Here, cell content refers to the content contained within a cell, such as text within the cell. For example, if the cell content of all three cells is text, then the text of the three cells is compared, and the first best cell content is selected. Specifically, there are three possible scenarios: In the first case, if the content of all three cells is the same, then the same text is taken as the first best cell content. Since all three cell contents are the best content, the HTML tags do not need to be updated. Also, since all three cell contents are the best content, the selection count of all three models can be incremented by one, indicating that for the second current cell, all three models have identified the best answer.

[0081] In the second scenario, if two of the three cells have the same content, this identical text is selected as the first optimal cell content, and it is used to update the first HTML tag, the updated second HTML tag, and the updated third HTML tag. This process ensures that the first HTML tag, the updated second HTML tag, and the updated third HTML tag all contain cell content with higher credibility. Simultaneously, the selection count of the models corresponding to the two cells with the same text is incremented by one. For example, consider cells A, B, and C. If cells A and B have the same text, this identical text is updated in the first HTML tag, the updated second HTML tag, and the updated third HTML tag. Furthermore, assuming cell A is a cell in the intersection set and cell B is a cell in the first cell set, since the intersection corresponds to the first HTML tag, the first HTML tag corresponds to the first model, the first cell set corresponds to the second HTML tag, and the second HTML tag corresponds to the second model, the selection count of both the first and second models can be incremented by one.

[0082] In the third scenario, if the contents of the three cells are all different, the cell region image can be cropped from the original image based on the cell coordinate information output by the first model. An independent OCR engine is then called to perform text recognition on that cell, obtaining the recognition result. This recognition result is then updated in the first HTML tag, the updated second HTML tag, and the updated third HTML tag. Next, it is checked whether the contents of the three cells are the same as the recognition result. If the contents of any cell are the same as the recognition result, the selection count of the model corresponding to that cell is incremented by one.

[0083] S503. Based on the first difference set, the first cell set, and the priority, elect and update the labels of the cells in the first difference set, and update the number of selections of each model according to the update results.

[0084] As an optional implementation, the cells in the first difference set are traversed. For the third current cell, the third comparison cell corresponding to the third current cell in the first cell set is determined. The second best cell content in the third current cell and the third comparison cell is determined according to the priority. The second best cell content is updated in the first HTML tag and the updated second HTML tag. The selection count of the model corresponding to the cell to which the second best cell content belongs is incremented by one.

[0085] Optionally, the third current cell corresponds to the first model, and the third comparison cell corresponds to the second model. Therefore, based on the priority of the first and second models, the second best cell content can be determined. For example, if the priority of the first model is higher than that of the second model, the content of the third current cell is taken as the second best cell content. Then, the second best cell content is updated in the first HTML tag and the updated second HTML tag. Furthermore, the selection count of the first model is incremented by one.

[0086] S504. Based on the second difference set, the second cell set, and the priority, elect and update the labels of the cells in the second difference set, and determine the number of times each model is selected based on the update results.

[0087] The execution process of this step is the same as that of step S503 above, and can be referred to step S503 above. It will not be repeated here.

[0088] S505. Determine the target HTML tag based on the number of times each model is selected.

[0089] Specifically, the HTML tag corresponding to the model with the most selections is used as the target HTML tag. For example, if the second model has the most selections, then the updated second HTML tag is used as the target HTML tag. It should be understood that the updated second HTML tag is the HTML tag that has been updated after the aforementioned steps S501-S504 have been performed.

[0090] In this embodiment, cell election and model election are performed sequentially based on the intersection, the first difference, and the second difference, which can make the accuracy of the obtained target HTML tags higher.

[0091] The following describes the process of obtaining the annotation results of the target table based on the target HTML tags in step S204.

[0092] In one approach, the annotation results of the target table can be obtained by manually checking and correcting the annotation based on the highlight attributes in the target HTML tags.

[0093] In another approach, the target HTML tags can be analyzed based on a multimodal large model. The annotation results of the target table can be obtained based on the analysis results and manual verification and correction.

[0094] For example, the multimodal large model could be gp t4o, qwen235b, etc. The original image containing the target table, the target HTML tags, and the following prompts are input into the multimodal large model to obtain analysis results. The analysis results include cell row numbers, column numbers, error types, prediction results, and the correct results determined by the large model.

[0095] Example of a prompt word: The input image is the original table from the financial notes scenario, and the input text is the HTML of the financial notes scenario table. The function checks the Chinese account names in the HTML; if errors are found, it outputs the row number, column number, error type, original text, prediction result, and the correct result determined by the large model. For example: { "1,1": "The original text is semantically correct, the original text = minus: accumulated depreciation, the predicted result = minus: accumulated depreciation, the correct result = minus: accumulated depreciation", "10,10": "The original text is semantically incorrect. Original text = no significant impact. Prediction result = no significant impact. Correct = no significant impact." } If there are no errors, an empty JSON is returned.

[0096] The analysis results output by the multimodal large model are parsed to obtain the row and column numbers of semantically correct but incorrectly predicted cells in the original image. Based on this, the cells are manually checked. If the cell or any character within it has a highlighted attribute, the result is manually confirmed as correct, and the cells or characters with highlighted attributes in the target HTML tag are updated. If there are rows highlighted in red in the target HTML tag, the labeling of those rows is manually checked for accuracy. If not, the result is manually confirmed as correct, and the red-highlighted rows in the target HTML tag are updated.

[0097] Tables 1, 2, and 3 below show the verification results of the scheme in this application.

[0098] Table 1. Correct Answer Rate

[0099] Table 2 Semantic Validation

[0100] Table 3 Accuracy Improvement

[0101] Based on the same inventive concept, this application also provides a table data annotation device corresponding to the table data annotation method. Since the principle of the device in this application is similar to the table data annotation method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0102] Figure 6 A modular structure diagram of the tabular data annotation device provided in this application is shown below. Figure 6 As shown, the device includes: The generation module 601 is used to generate a first HTML tag corresponding to the first model based on the recognition result of the first model for the target table, generate a second HTML tag corresponding to the second model based on the recognition result of the second model for the target table, and generate a third HTML tag corresponding to the third model based on the recognition result of the third model for the target table.

[0103] The first comparison module 602 is used to perform a difference comparison between the first HTML tag and the second HTML tag, determine the first comparison information of the second HTML tag relative to the first HTML tag, and update the second HTML tag according to the first comparison information to obtain the updated second HTML tag.

[0104] The second comparison module 603 is used to perform a difference comparison between the first HTML tag and the third HTML tag, determine the second comparison information of the third HTML tag relative to the first HTML tag, and update the third HTML tag according to the second comparison information to obtain the updated third HTML tag.

[0105] The annotation module 604 is used to perform election and tag update based on the first HTML tag, the updated second HTML tag, and the updated second HTML tag to obtain the target HTML tag, and to obtain the annotation result of the target table based on the target HTML tag.

[0106] As an optional implementation, the generation module 601 is specifically used for: The recognition results of the first model for the target table are standardized to obtain the first HTML tag. The first HTML tag includes at least: row identifier, row placeholder, column identifier, column placeholder, cell identifier, cell content, and cell coordinate information.

[0107] As an optional implementation, the first comparison module 602 is specifically used for: The first HTML tag and the second HTML tag are compared by row, cell, and character to obtain the first comparison information, which includes: row comparison result, cell comparison result, and character comparison result; Based on the first comparison information, a highlight attribute is set for the target element in the second HTML tag to obtain the updated second HTML tag. The target element includes a row, cell, or character.

[0108] As an optional implementation, the first comparison module 602 is specifically used for: Determine the line comparison result of the i-th line in the second HTML tag relative to the first HTML tag. The line comparison result is used to indicate the editing status of the i-th line in the second HTML tag relative to the j-th line of the first HTML tag. The editing status includes: modified, redundant, or consistent. If the row i in the second HTML tag is modified relative to the row comparison result of the first HTML tag, and the row i-1 in the second HTML tag is consistent with the row comparison result of the first HTML tag, and the row i+1 in the second HTML tag is consistent with the row comparison result of the first HTML tag, then the row p in the row i in the second HTML tag is determined to be consistent with the row j in the first HTML tag. The row comparison result is used to indicate the editing status of the row p in the row i in the second HTML tag relative to the row q in the row j in the first HTML tag. If the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell in the j-th row of the first HTML tag is modified, then the character comparison result of the m-th character in the p-th cell in the i-th row of the second HTML tag with the q-th cell in the j-th row of the first HTML tag is determined. The character comparison result is used to indicate the editing status of the m-th character in the p-th cell in the i-th row of the second HTML tag with the n-th character in the q-th cell in the j-th row of the first HTML tag.

[0109] As an optional implementation, the first comparison module 602 is specifically used for: If the line of the i-th row in the second HTML tag is redundant relative to the line comparison result of the first HTML tag, then the first highlight attribute is set for the line of the i-th row in the second HTML tag; If the comparison result of the i-th row in the second HTML tag with the row of the first HTML tag is modified, and the comparison result of the (i-1)-th row in the second HTML tag with the row of the first HTML tag is consistent, and the comparison result of the (i+1)-th row in the second HTML tag with the row of the first HTML tag is consistent; if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is redundant, then set the p-th cell in the i-th row of the second HTML tag with a second highlight attribute; if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is modified, then set the p-th cell in the i-th row of the second HTML tag with a third highlight attribute.

[0110] As an optional implementation, the annotation module 604 is specifically used for: Based on the baseline evaluation results of the first model, the second model, and the third model, the priorities of the first model, the second model, and the third model are determined. Based on the cells with highlight attributes in the updated second HTML tag, determine the first set of cells corresponding to the updated second HTML tag, and determine the first set of mapped cells corresponding to the first HTML tag based on the first set of cells, where each cell in the first set of cells corresponds to a cell in the first set of mapped cells. Based on the cells with highlight attributes in the updated third HTML tag, determine the second set of cells corresponding to the updated third HTML tag, and determine the second set of mapped cells corresponding to the first HTML tag based on the second set of cells. Each cell in the second set of cells corresponds to a cell in the second set of mapped cells. Based on the priority, the first cell set, the first mapped cell set, the second cell set, and the second mapped cell set, an election and tag update are performed to obtain the target HTML tag.

[0111] As an optional implementation, the annotation module 604 is specifically used for: Add the cells with the highlight attribute in the updated second HTML tag to the first cell set; Iterate through each cell in the first cell set. For the first current cell, determine the corresponding cell in the first HTML tag based on the cell comparison result of the first current cell. Add the corresponding cell to the first mapped cell set and establish the mapping relationship between the first current cell and the corresponding cell.

[0112] As an optional implementation, the annotation module 604 is specifically used for: Determine the intersection of the first set of mapped cells and the second set of mapped cells, the first difference between the first set of mapped cells and the second set of mapped cells, and the second difference between the second set of mapped cells and the first set of mapped cells; Based on the intersection, the first set of cells, and the second set of cells, the cells in the intersection are selected and their labels are updated, and the number of times each model is selected is determined based on the update results. Based on the first difference set, the first cell set, and the priority, the cells in the first difference set are selected and their labels are updated, and the number of selections for each model is updated based on the update results. Based on the second difference set, the second cell set, and the priority, the cells in the second difference set are selected and their labels are updated, and the number of selections for each model is updated based on the update results. The target HTML tag is determined based on the number of times each model is selected.

[0113] As an optional implementation, the annotation module 604 is specifically used for: Traverse each cell in the intersection, and for the second current cell encountered, determine the first comparison cell corresponding to the second current cell in the first cell set, and determine the second comparison cell corresponding to the second current cell in the second cell set; The cell contents of the second current cell, the first comparison cell, and the second comparison cell are compared to obtain the first best cell content. The first best cell content is then updated in the first HTML tag, the updated second HTML tag, and the updated third HTML tag. Based on the cell to which the first best cell content belongs, the selection count of the model corresponding to that cell is incremented by one.

[0114] As an optional implementation, the annotation module 604 is specifically used for: Traverse each cell in the first difference set, and for the third current cell encountered, determine the third comparison cell corresponding to the third current cell in the first cell set; Based on the priority, determine the second best cell content in the third current cell and the third comparison cell, update the second best cell content in the first HTML tag and the updated second HTML tag, and increment the selection count of the model corresponding to the cell to which the second best cell content belongs by one.

[0115] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0116] This application also provides an electronic device 70, such as... Figure 7 The diagram shown is a structural schematic of an electronic device 70 provided in an embodiment of this application, including: a processor 71 and a memory 72. Optionally, it may also include a bus 73. The memory 72 stores machine-readable instructions executable by the processor 71. When the electronic device 70 is running, the processor 71 communicates with the memory 72 via the bus 73, and the processor 71 executes the machine-readable instructions to perform the steps of the above-described table data annotation method.

[0117] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described table data annotation method.

[0118] This application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-described table data annotation method.

[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0121] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for labeling tabular data, characterized in that, include: Based on the recognition result of the first model for the target table, generate the first HTML tag corresponding to the first model; based on the recognition result of the second model for the target table, generate the second HTML tag corresponding to the second model; based on the recognition result of the third model for the target table, generate the third HTML tag corresponding to the third model. The difference between the first HTML tag and the second HTML tag is compared to determine the first comparison information of the second HTML tag relative to the first HTML tag, and the second HTML tag is updated according to the first comparison information to obtain the updated second HTML tag; The first HTML tag and the third HTML tag are compared to determine the second comparison information of the third HTML tag relative to the first HTML tag, and the third HTML tag is updated according to the second comparison information to obtain the updated third HTML tag; Based on the first HTML tag, the updated second HTML tag, and the updated second HTML tag, an election and tag update are performed to obtain the target HTML tag, and the annotation result of the target table is obtained based on the target HTML tag.

2. The method according to claim 1, characterized in that, The step of generating the first Hypertext Markup Language (HTML) tag corresponding to the first model based on the recognition result of the first model for the target table includes: The recognition results of the first model for the target table are standardized to obtain the first HTML tag. The first HTML tag includes at least: row identifier, row placeholder, column identifier, column placeholder, cell identifier, cell content, and cell coordinate information.

3. The method according to claim 1, characterized in that, The step of comparing the first HTML tag and the second HTML tag to determine the first comparison information of the second HTML tag relative to the first HTML tag, and updating the second HTML tag according to the first comparison information to obtain the updated second HTML tag, includes: The first HTML tag and the second HTML tag are compared by row, cell, and character to obtain the first comparison information, which includes: row comparison result, cell comparison result, and character comparison result; Based on the first comparison information, a highlight attribute is set for the target element in the second HTML tag to obtain the updated second HTML tag. The target element includes a row, cell, or character.

4. The method according to claim 3, characterized in that, The step of performing row comparison, cell comparison, and character comparison between the first HTML tag and the second HTML tag to obtain the first comparison information includes: Determine the line comparison result of the i-th line in the second HTML tag relative to the first HTML tag. The line comparison result is used to indicate the editing status of the i-th line in the second HTML tag relative to the j-th line of the first HTML tag. The editing status includes: modified, redundant, or consistent. If the row i in the second HTML tag is modified relative to the row comparison result of the first HTML tag, and the row i-1 in the second HTML tag is consistent with the row comparison result of the first HTML tag, and the row i+1 in the second HTML tag is consistent with the row comparison result of the first HTML tag, then the row p in the row i in the second HTML tag is determined to be consistent with the row j in the first HTML tag. The row comparison result is used to indicate the editing status of the row p in the row i in the second HTML tag relative to the row q in the row j in the first HTML tag. If the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell in the j-th row of the first HTML tag is modified, then the character comparison result of the m-th character in the p-th cell in the i-th row of the second HTML tag with the q-th cell in the j-th row of the first HTML tag is determined. The character comparison result is used to indicate the editing status of the m-th character in the p-th cell in the i-th row of the second HTML tag with the n-th character in the q-th cell in the j-th row of the first HTML tag.

5. The method according to claim 3, characterized in that, The step of setting a highlight attribute for the target element in the second HTML tag based on the first comparison information to obtain the updated second HTML tag includes: If the line of the i-th row in the second HTML tag is redundant relative to the line comparison result of the first HTML tag, then the first highlight attribute is set for the line of the i-th row in the second HTML tag; If the comparison result of the i-th row in the second HTML tag with the row of the first HTML tag is modified, and the comparison result of the (i-1)-th row in the second HTML tag with the row of the first HTML tag is consistent, and the comparison result of the (i+1)-th row in the second HTML tag with the row of the first HTML tag is consistent; if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is redundant, then set the p-th cell in the i-th row of the second HTML tag with a second highlight attribute; if the comparison result of the p-th cell in the i-th row of the second HTML tag with the cell of the j-th row of the first HTML tag is modified, then set the p-th cell in the i-th row of the second HTML tag with a third highlight attribute.

6. The method according to claim 1, characterized in that, The step of selecting and updating tags based on the first HTML tag, the updated second HTML tag, and the updated second HTML tag to obtain the target HTML tag includes: Based on the baseline evaluation results of the first model, the second model, and the third model, the priorities of the first model, the second model, and the third model are determined. Based on the cells with highlight attributes in the updated second HTML tag, determine the first set of cells corresponding to the updated second HTML tag, and determine the first set of mapped cells corresponding to the first HTML tag based on the first set of cells, where each cell in the first set of cells corresponds to a cell in the first set of mapped cells. Based on the cells with highlight attributes in the updated third HTML tag, determine the second set of cells corresponding to the updated third HTML tag, and determine the second set of mapped cells corresponding to the first HTML tag based on the second set of cells. Each cell in the second set of cells corresponds to a cell in the second set of mapped cells. Based on the priority, the first cell set, the first mapped cell set, the second cell set, and the second mapped cell set, an election and tag update are performed to obtain the target HTML tag.

7. The method according to claim 6, characterized in that, The step of determining the first set of cells corresponding to the updated second HTML tag based on the cells with the highlighted attribute in the updated second HTML tag, and determining the first set of mapped cells corresponding to the first HTML tag based on the first set of cells, includes: Add the cells with the highlight attribute in the updated second HTML tag to the first cell set; Iterate through each cell in the first cell set. For the first current cell, determine the corresponding cell in the first HTML tag based on the cell comparison result of the first current cell. Add the corresponding cell to the first mapped cell set and establish the mapping relationship between the first current cell and the corresponding cell.

8. The method according to claim 6, characterized in that, The process of electing and updating tags based on the priority, the first cell set, the first mapped cell set, the second cell set, and the second mapped cell set to obtain the target HTML tag includes: Determine the intersection of the first set of mapped cells and the second set of mapped cells, the first difference between the first set of mapped cells and the second set of mapped cells, and the second difference between the second set of mapped cells and the first set of mapped cells; Based on the intersection, the first set of cells, and the second set of cells, the cells in the intersection are selected and their labels are updated, and the number of times each model is selected is determined based on the update results. Based on the first difference set, the first cell set, and the priority, the cells in the first difference set are selected and their labels are updated, and the number of selections for each model is updated based on the update results. Based on the second difference set, the second cell set, and the priority, the cells in the second difference set are selected and their labels are updated, and the number of selections for each model is updated based on the update results. The target HTML tag is determined based on the number of times each model is selected.

9. The method according to claim 8, characterized in that, The step of selecting and updating labels for cells in the intersection, the first cell set, and the second cell set, and determining the number of selections for each model based on the update results, includes: Traverse each cell in the intersection, and for the second current cell encountered, determine the first comparison cell corresponding to the second current cell in the first cell set, and determine the second comparison cell corresponding to the second current cell in the second cell set; The cell contents of the second current cell, the first comparison cell, and the second comparison cell are compared to obtain the first best cell content. The first best cell content is then updated in the first HTML tag, the updated second HTML tag, and the updated third HTML tag. Based on the cell to which the first best cell content belongs, the selection count of the model corresponding to that cell is incremented by one.

10. The method according to claim 8, characterized in that, The step of electing and updating the labels of cells in the first difference set based on the first difference set, the first cell set, and the priority, and updating the selection count of each model based on the update results, includes: Traverse each cell in the first difference set, and for the third current cell encountered, determine the third comparison cell corresponding to the third current cell in the first cell set; Based on the priority, determine the second best cell content in the third current cell and the third comparison cell, update the second best cell content in the first HTML tag and the updated second HTML tag, and increment the selection count of the model corresponding to the cell to which the second best cell content belongs by one.

11. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when the electronic device is running, are executed by the processor to perform the steps of the tabular data annotation method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the tabular data annotation method according to any one of claims 1 to 10.

13. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the tabular data annotation method according to any one of claims 1 to 10.

Images