A table image conversion method and device

Abstract

This application discloses a method and apparatus for converting table images, specifically relating to the field of computer technology. The table image conversion method includes: converting a target table image using a target model to obtain a Hypertext Markup Language (HMR) conversion result; wherein the target model is a model obtained after training and adjusting the grammatical layer reward value calculated from the HMR sequence based on a hierarchical grammatical reward function, and the HMR sequence is obtained by converting a sample table image using a pre-set model before adjustment. This application improves the grammatical correctness of the HMR sequence generated by the model by using a target model obtained after training and adjusting the hierarchical grammatical reward function to convert the table image, enabling the target model to generate grammatically correct and structurally sound HMR conversion results. This improves the usability of the table image conversion results.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method and apparatus for converting table images. Background Technology

[0002] Tables, as an important carrier of structured information, are widely used in corporate documents, reports, financial statements, and other fields. In practical applications, many tables exist in image form, but table images cannot be directly parsed and edited by computers. They need to be converted into HyperText Markup Language (HTML) format to achieve document digitization and information extraction.

[0003] Related technologies typically employ general multimodal large models to perform the task of converting tabular images into hypertext markup language. However, general multimodal large models often produce outputs with syntax errors, rendering the conversion results unusable. Summary of the Invention

[0004] A table-image conversion method and apparatus are provided to solve the technical problem that syntax errors in the output of general multimodal large models lead to unusable conversion results.

[0005] In a first aspect, embodiments of this application provide a table image conversion method, comprising the following steps: The target table image is converted using a target model to obtain the Hypertext Markup Language conversion result; The target model is a model obtained by training and adjusting the grammatical layer reward value calculated on the hypertext markup language sequence based on the hierarchical grammatical reward function. The hypertext markup language sequence is obtained by converting the sample table image using the preset model before adjustment.

[0006] In one embodiment of this application, the hierarchical grammar reward function includes a label hierarchy, an attribute hierarchy, and a structural hierarchy; The grammar layer reward value is obtained by weighted summation of the tag pairing correctness score, the attribute usage normativity score, and the table structure integrity score, wherein the tag pairing correctness score is the quality assessment result of the hypertext markup language sequence at the tag level, the attribute usage normativity score is the quality assessment result of the hypertext markup language sequence at the attribute level, and the table structure integrity score is the quality assessment result of the hypertext markup language sequence at the structure level.

[0007] In one embodiment of this application, the generation quality assessment of the hypertext markup language sequence based on the tag hierarchy includes: Traverse the hypertext markup language sequence, extract all tags and verify them, and record the number of tag matching errors; The tag matching accuracy score is obtained based on the number of tag matching errors and the total number of tags.

[0008] In one embodiment of this application, the step of extracting all tags and verifying them includes: Push the start label onto the stack and verify the match between the end label and the start label at the top of the stack. If there is no match or the stack is empty, record the label pairing error. Verify the nesting rules of the tags, and record tag pairing errors if the preset nesting rules are violated.

[0009] In one embodiment of this application, the generation quality assessment of the hypertext markup language sequence based on the attribute hierarchy includes: Perform attribute validation on the attributes of the tag and obtain the attribute validation results; Based on the attribute verification results, a weighted calculation is performed according to the preset attribute value weights to obtain the attribute usage standardization score.

[0010] In one embodiment of this application, the attribute verification includes attribute name validity verification, attribute value validity verification, and attribute combination rationality verification.

[0011] In one embodiment of this application, the generation quality assessment of the hypertext markup language sequence based on the structural hierarchy includes: A virtual grid matrix is ​​constructed to record the cell occupancy at each position in the hypertext markup language sequence. Matrix integrity is verified based on the virtual grid matrix to obtain a matrix integrity score. Verify whether there are any conflicts in the merged cells, and obtain a merge validity score based on the number of conflicting cells and the total number of cells; Perform a content integrity check on the content of each cell and obtain a content integrity score; The table structure integrity score is obtained based on the matrix integrity score, the merge validity score, and the content integrity score.

[0012] In one embodiment of this application, the table image conversion method further includes: For the same sample table image, G hypertext markup language sequences are generated independently using the preset model; where G is a positive integer.

[0013] In one embodiment of this application, the table image conversion method further includes: Based on the grammar layer reward value, the average reward of all sequences and the standard deviation of the rewards of all sequences are obtained to obtain the relative advantage value; Based on the relative advantage value, the target optimization value of the reinforcement learning method is obtained, and the preset model is adjusted using the reinforcement learning method to obtain the target model.

[0014] Secondly, embodiments of this application also provide a table image conversion apparatus, which is used to convert a target table image using a target model to obtain a Hypertext Markup Language conversion result; The target model is a model obtained by training and adjusting the grammatical layer reward value calculated on the hypertext markup language sequence based on the hierarchical grammatical reward function. The hypertext markup language sequence is obtained by converting the sample table image using the preset model before adjustment.

[0015] The beneficial effects of this application are as follows: This application uses a target model trained and adjusted using a hierarchical grammatical reward function to convert table images. During the training phase, the target model optimizes the grammatical layer reward value calculated based on the hierarchical grammatical reward function for the hypertext markup language sequence, thereby improving the grammatical correctness of the generated hypertext markup language sequence. This enables the target model to generate grammatically correct and structurally sound hypertext markup language conversion results. This effectively solves the problem that general multimodal large models often produce grammatical errors in table image conversion, leading to unusable conversion results, and improves the usability of table image conversion results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the steps for obtaining a tag pairing correctness score according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the steps of obtaining a normative score for an attribute according to an embodiment of this application; Figure 3 This is a schematic diagram illustrating the steps for obtaining a table structure integrity score according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating the steps of adjusting the preset model in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] Embodiments of this application provide a table image conversion method, such as... Figure 1 As shown, it includes the following steps: The target table image is converted using a target model to obtain the Hypertext Markup Language conversion result; The target model is a model obtained by training and adjusting the grammatical layer reward value calculated on the hypertext markup language sequence based on the hierarchical grammatical reward function. The hypertext markup language sequence is obtained by converting the sample table image using the preset model before adjustment.

[0021] This application embodiment uses a target model trained and adjusted by a hierarchical grammatical reward function to convert table images. During the training phase, the target model optimizes the grammatical layer reward value calculated by the hierarchical grammatical reward function for the hypertext markup language sequence, thereby improving the grammatical correctness of the hypertext markup language sequence generated by the model. This enables the target model to generate grammatically correct and structurally sound hypertext markup language conversion results, effectively solving the problem that general multimodal large models often produce grammatical errors in table image conversion, resulting in unusable conversion results, and improving the usability of table image conversion results.

[0022] In one embodiment of this application, the hierarchical grammar reward function includes a label hierarchy, an attribute hierarchy, and a structural hierarchy; The grammatical layer reward value is obtained by weighted summation of the tag matching correctness score, attribute usage normativity score, and table structure integrity score. The tag matching correctness score is the result of the hypertext markup language sequence quality assessment at the tag level, the attribute usage normativity score is the result of the hypertext markup language sequence quality assessment at the attribute level, and the table structure integrity score is the result of the hypertext markup language sequence quality assessment at the structure level.

[0023] To address the technical problem that general multimodal large models often produce outputs with grammatical errors such as label mismatch, invalid attribute values, and incomplete structure, this application embodiment constructs a hierarchical grammatical reward function to evaluate the generation quality of hypertext markup language sequences at the label level, attribute level, and structural level, and obtains grammatical layer reward values.

[0024] The grammatical layer reward value is obtained by weighted summation of the tag matching correctness score, attribute usage normativity score, and table structure integrity score. The tag matching correctness score is the result of the hypertext markup language sequence quality assessment at the tag level, the attribute usage normativity score is the result of the hypertext markup language sequence quality assessment at the attribute level, and the table structure integrity score is the result of the hypertext markup language sequence quality assessment at the structure level.

[0025] The hierarchical grammar reward function evaluates the generation quality at the tag level, attribute level, and structure level, allowing the model to improve corresponding grammatical problems in a targeted manner based on the evaluation results at the tag level, attribute level, and structure level.

[0026] In some embodiments, the hierarchical grammar reward function R syntax The expression is: ; Among them, T balance ω1 represents the label matching correctness score, and A represents the weight of the label hierarchy. valid This indicates that the attribute uses a normalized score, ω2 represents the weight of the attribute hierarchy, and S complete ω3 represents the score for the integrity of the table structure, and ω3 represents the weight of the structural hierarchy.

[0027] Hypertext Markup Language (HTML) requires that all tags, except for self-closing tags, must appear in pairs and be correctly nested. Tag pairing errors are the most common syntax problem in generating HTML sequences of tables. For example, a financial statement containing complex merged cells might require generating hundreds of tags; a single incorrect tag pairing will prevent the entire table from rendering correctly. Directly relying on the model to autoregressively generate HTML sequence code lacks explicit constraints on tag pairing. The model may forget previously opened tags during generation, leading to unclosed or incorrectly closed tags. This is especially problematic when dealing with deeply nested structures, such as tables within tables, where the tag pairing error rate rises sharply.

[0028] To address the technical issues mentioned above, this application guides the model to generate correctly paired and compliant Hypertext Markup Language sequences by verifying the correctness of tag pairings.

[0029] In some embodiments, such as Figure 1 As shown, the generation quality assessment of Hypertext Markup Language sequences based on tag hierarchy includes: Step A1: Traverse the Hypertext Markup Language sequence, extract all tags and verify them, and record the number of tag matching errors; Step A2: Obtain a tag matching accuracy score based on the number of incorrect tag matchings and the total number of tags.

[0030] In some embodiments, the label matching correctness score T balance The expression is: ; Among them, T balance σ represents the label matching correctness score; σ represents the activation function, used to compress continuous values ​​of the input into the (0, 1) interval; β T This represents the label layer temperature parameter, used to adjust the output sensitivity of the activation function, β. T The larger the β value, the more sensitive the score is to changes in the error rate. T The smaller the value of N, the more gradual the score change; tag_error N represents the number of incorrect label pairings. total_tags This indicates the total number of tags.

[0031] In step A1, all tags are extracted and verified, including: Push the start label onto the stack and verify the match between the end label and the start label at the top of the stack. If there is no match or the stack is empty, record the label matching error.

[0032] Verify the tag nesting rules and log tag pairing errors if the preset nesting rules are violated.

[0033] Self-closing tags do not require pairing.

[0034] This application embodiment uses a stack-based tag verification method to count the number of tag pairing errors. The tag verification scenarios in this application include basic pairing verification, nested level verification, and self-closing tag processing.

[0035] Under basic pairing verification, an empty stack is initialized. The generated Hypertext Markup Language sequence is traversed, and all tags are extracted. When a start tag is encountered, it is pushed onto the stack. When an end tag is encountered, it is checked whether the stack is empty and whether the top tag of the stack matches the current end tag. If there is no match or the stack is empty, a tag pairing error is recorded.

[0036] Because table tags must follow specific nesting rules, for example, row tags must be table tags.

[0037] <input>

[0038]

[0039]

[0040] Figure 2

[0041]

[0042]

[0043]

[0044] Header Labels Table label Or the label at the end of the table direct child elements; cell tags (regular cell) (Header cells) must be row labels. The direct child element of the table. Even if the tags are correctly paired, errors must still be logged if nesting is violated. Self-closing tags such as line break tags, image tags, and input box tags do not need to be paired, but they are rarely used in table contexts. The following table, shown in Table 1, serves as an example: Table 1: Table Example. The correct use of Hypertext Markup Language tag attributes is crucial for table rendering. Each tag has its specific set of attributes, and attribute values ​​must conform to specific format requirements. For example, column merging and row merging must be positive integers, and alignment attributes can only take specific enumerated values. Due to a lack of understanding of attribute specifications in related models, invalid attributes or incorrect attribute values ​​are frequently generated, including: using non-existent attribute names, incorrect attribute value formats, and contradictory attribute combination logic. While these errors may not necessarily cause page crashes, they will affect the correct display and semantic expression of the table. To address the aforementioned technical problems, this application guides the model to generate correct tag attributes by verifying the standardization of Hypertext Markup Language attribute usage. In some embodiments, as shown, the generation quality assessment of the Hypertext Markup Language sequence based on attribute hierarchy includes: Step B1, performing attribute verification on the attributes of the tags to obtain attribute verification results; attribute verification includes attribute name validity verification, attribute value validity verification, and attribute combination rationality verification. Step B2, based on the attribute verification results, performing weighted calculation according to preset attribute value weights to obtain an attribute usage standardization score. In some embodiments, the attribute verification in Step B1 includes attribute name validity verification, attribute value validity verification, and attribute combination rationality verification. Each Hypertext Markup Language tag has its allowed attribute set. Attribute name validity verification is used to verify that the attribute used by the tag belongs to its preset allowed attribute set. In this embodiment, the generated Hypertext Markup Language sequence is traversed, the attribute name of each tag is extracted, and compared with the attribute dictionary. If the attribute name is not in the allowed set of the corresponding tag, it is determined that the attribute name is illegal. For example, the attribute dictionary is as follows: Table Tags

[0045]

[0046] Allowed attributes include: border width, cell padding, cellspacing, width, height, horizontal alignment, background color, inline style, class name, and unique identifier (id). Table row labels. Allowed attributes include: horizontal alignment (align), vertical alignment (valign), background color (bgcolor), inline style (style), class name (class), and unique identifier (id). Table cell tags Or header cell labels Allowed attributes include: colspan (number of columns), rowspan (number of rows), horizontal alignment (align), vertical alignment (valign), width (width), height (height), background color (bgcolor), inline style (style), class name (class), and unique identifier (id); among these, the scope attribute is only allowed in the header cell label. Used in.

[0047] For complex tables, the correct use of attributes is especially important. Take Table 2, which contains merged cells, as an example: Table 2: Example table containing merged cells

[0048] Because specific value standards are established for key table attributes, the value range requirements differ for different attributes. This embodiment uses attribute value validity validation to verify that the attribute value corresponding to a valid attribute name conforms to preset value rules, avoiding errors in value type or range. For example, the value range requirements for different attributes are as follows: The number of columns (colspan) or rows (rowspan) must be a positive integer, and the value usually does not exceed the number of columns or rows in the table.

[0049] The horizontal alignment option (align) can take the following values: left alignment, right alignment, center alignment, and justify alignment.

[0050] The vertical alignment (valign) can take the following values: top, middle, bottom, and baseline.

[0051] The width or height can be a pixel value, such as 100 or 100px; or a percentage value, such as 50%.

[0052] The border width (border) value must be a non-negative integer.

[0053] In some embodiments, attribute value validity validation uses a validation function V. attr (a,v) checks each attribute value and converts the compliance level of the attribute value into a quantitative score, as shown in the following expression: ; Where v represents the attribute value; v∈ValidRange(a) means completely valid, and gets 1 point; v issemantically acceptable means acceptable, and gets 0.5 points; otherwise means the attribute value is completely invalid, and gets 0 points.

[0054] Some attribute combinations may be syntactically correct but logically contradictory. For example, if a cell's colspan value exceeds the total number of columns in the table, although the Hypertext Markup Language (HTML) will not report an error, it will exceed the table's boundaries during rendering, which is clearly unreasonable. This embodiment detects such problems through context analysis. For example, it verifies whether the attribute value conflicts with the total number of rows or columns in the table or with other cell attributes; if a contradiction exists, the score for that attribute is directly recorded as 0.

[0055] In some embodiments, the attribute uses a normative score A. valid The expression is: ; Where, N attr Indicates the total number of attributes; w i This indicates the importance weight of the i-th attribute. For example, attributes that affect the table structure, such as the number of columns spanned (colspan) and the number of rows spanned (rowspan), need to be assigned higher weights.

[0056] A grammatically correct Hypertext Markup Language (HTML) requires not only correct tags and attributes but also structural integrity and consistency. For example, each row should have the same number of columns, merged cells should not exceed table boundaries, and nested structures should conform to HTML specifications. Related models often focus only on local tag generation, lacking global structural constraints. This results in tables that appear logical in parts but have overall structural flaws. For instance, the actual number of columns in a row may not match the number of columns in the header, or the cross-row / cross-column values ​​of merged cells may exceed the table's boundaries.

[0057] To address the aforementioned technical problems, this application optimizes the structural hierarchy of a preset model by verifying the integrity of the table structure. In some embodiments, such as... Figure 3 As shown, the generation quality assessment of Hypertext Markup Language sequences based on structural hierarchy includes: Step C1: Construct a virtual grid matrix to record the cell occupancy at each position in the Hypertext Markup Language sequence. Perform matrix integrity verification based on the virtual grid matrix to obtain a matrix integrity score. Step C2: Verify whether there are any conflicts in the merged cells, and obtain a merge validity score based on the number of conflicting cells and the total number of cells; Step C3: Perform a content integrity check on the content of each cell and obtain a content integrity score; Step C4: Obtain the table structure integrity score based on the matrix integrity score, merge validity score, and content integrity score.

[0058] In this embodiment, verifying the integrity of the table structure includes three aspects: verifying matrix integrity, merging validity, and content integrity.

[0059] Matrix integrity verification is used to check whether a table has a complete matrix structure. Ideally, a table should form a complete matrix structure. This example considers the impact of merged cells and constructs a virtual grid matrix G, expressed as: ; Where r represents the total number of rows in the table, c represents the total number of columns in the table, and N represents the total number of positions in the matrix, which is the product of the total number of rows r and the total number of columns c.

[0060] For example, for the following table structure: Row 1: [Cell 1(colspan=2)] [Cell 2] Row 2: [Cell 3] [Cell 4(colspan=2)] Row 3: [Cell 5] [Cell 6] [Cell 7] The corresponding grid matrix is: ; The integrity check in this embodiment must ensure that it meets the following rules: Each position is occupied by exactly one cell, with no gaps.

[0061] Merged cells occupy a contiguous rectangular area.

[0062] After considering the column spanning property of cells, the number of valid columns in each row is equal.

[0063] In some embodiments, the matrix integrity score M complete The expression is: ; Where |(i,j):G_ij≠0| represents the number of non-empty positions in the matrix, i.e., the number of cells occupied, N represents the total number of positions in the matrix, and M represents the matrix integrity score. complete For example, its value is between 0 and 1. The closer it is to 1, the fewer the holes and the more complete the matrix structure.

[0064] Merge validity validation is used to check whether merged cells meet physical constraints. For example, a cell that spans three rows (rowspan=3) must ensure that the corresponding positions in the two rows below it are not occupied by other cells.

[0065] For any cell k, its position is represented as (i k ,j k ), its span is (rs k ,cs k The range occupied by this cell is then represented as: ; This is represented as the cell occupying u∈[0,rs] in the matrix. k-1 ]、v∈[0,cs k-1 All positions of ], where rs k This indicates the number of rows spanned by the cell (rowspan), cs k This represents the number of columns spanned (colspan). If C is found during the update... ik+u , jk+v =1, meaning it is already occupied, then a conflict is determined.

[0066] This embodiment defines a conflict indication function δ. k : ; The conflict indicator function is used to quantify the result of whether a conflict has occurred into a numerical value. It outputs 1 if a conflict has occurred and 0 if no conflict has occurred.

[0067] In some embodiments, the combined validity score M merge The expression is: ; Where N represents the total number of cells, and M represents the merged validity score. merge The value range is from 0 to 1, and the combined validity score M merge The closer it is to 1, the fewer the cell conflicts are, which conforms to physical constraints.

[0068] Content integrity validation checks whether each cell contains content. A cell can be empty, but it must contain content of a valid type. For numeric tables, it checks the consistency of the numeric format. For text tables, it checks the correctness of the encoding.

[0069] In some embodiments, the content integrity score C content The expression is: ; Where rs represents the total number of rows in the table, cs represents the total number of columns in the table, N represents the number of cells in the table, and the content completeness score is C. content The value ranges from 0 to 1, with the value closer to 1 indicating that the cell content is more compliant.

[0070] in: ; For each cell position (i,j), verify the validity of its content. If the content is valid, V ij The output is 1; if the content is invalid, V ij The output is 0.

[0071] In some embodiments, based on the matrix integrity score M complete Merging validity score M merge Content integrity score C content Calculate the table structure integrity score S complete The expression is: ; Where α, β, and γ represent the weight coefficients of the sub-items.

[0072] Supervised learning can train multimodal large language models and enhance their ability to generate hypertext markup language sequences. However, supervised learning requires a large amount of labeled data and the improvement in performance is limited. On the other hand, directly using reinforcement learning, such as proximal policy optimization (PPO) or group-based relative policy optimization (GRPO), makes it difficult to obtain effective learning signals.

[0073] To address the aforementioned issues, this application adjusts the preset model using reinforcement learning based on the grammatical layer reward value. This solves both the problem of requiring a large amount of labeled data for supervised training of multimodal large language models and the technical problem of obtaining effective learning signals when using reinforcement learning.

[0074] In some embodiments, for the same sample table image, G hypertext markup language sequences are independently generated using a preset model; where G is a positive integer. By generating multiple candidate results of hypertext markup language sequences, the exploration space is expanded, and the probability of discovering high-quality solutions is increased.

[0075] Optimization is achieved through relative comparisons within a population of hypertext markup language sequences. Multiple outputs from the same input provide a natural comparison, facilitating the model's learning of which generation strategies are superior. Compared to absolute evaluation of a single generated hypertext markup language sequence, relative comparisons within a population are more stable and reduce the variance of gradient estimation.

[0076] In some embodiments, such as Figure 4 As shown, this application adjusts the preset model according to the grammatical layer reward value to obtain the target model, including the following steps: Step D1: Based on the grammar layer reward value, obtain the average reward of all sequences and the standard deviation of the reward of all sequences to obtain the relative advantage value; Step D2: Based on the relative advantage value, obtain the target optimization value of the reinforcement learning method, and adjust the preset model using the reinforcement learning method to obtain the target model.

[0077] This embodiment employs relative policy optimization of the population as a reinforcement learning method, eliminating the dependence of proximal policy optimization methods on the value function. For the same sample table image, a pre-defined model independently generates G hypertext markup language sequences; the expression for population relative advantage estimation is: ; Among them, R i This represents the original reward value, corresponding to the original reward of the i-th sample, which is the syntax layer reward value obtained through the reward function. This represents the average of all rewards within the current group. This represents the standard deviation of all rewards within the current group. This represents the relative advantage value obtained.

[0078] This estimation method directly utilizes the reward distribution within the group, transforming absolute rewards into relative advantages, which simplifies algorithm implementation and improves training stability.

[0079] The expression for the relative strategy optimization method for a population is: ; J GRPO (θ) represents the model parameters. The optimization objective is given by L(θ), where L(θ) is the strategy optimization term, i.e., the objective optimization value, βD. KL This represents the policy constraint. E denotes the expectation; (q,a) D represents the dataset, θ old Represents the parameters of the old model, β represents the KL divergence weights, and π represents the parameters of the old model. θ Indicates the new strategy, π ref Indicates the reference strategy, This represents a sequence of Hypertext Markup Language (HMRL) entries.

[0080] L(θ) is the target optimization value, based on the population relative advantage value obtained in this application. The expression is: ; Where G represents the number of Hypertext Markup Language sequences. Represents a Hypertext Markup Language sequence, ε represents the clipping factor, |o i | indicates the length of the Hypertext Markup Language sequence. Indicates the original contribution. This indicates the contribution after trimming.

[0081] For each Hypertext Markup Language (HText Markup) sequence i, the above expression calculates the original contribution and the pruned contribution. The minimum of the two is taken as the contribution value for that step. The contributions of all steps for HText Markup Language sequence i are summed, and then divided by the length of the HText Markup Language sequence |o. i |, obtain the normalized contribution. Sum the normalized contributions of all Hypertext Markup Language sequences, then divide by the population size G to finally obtain the policy optimization term L(θ).

[0082] In conjunction with the above embodiments, this application constructs a hierarchical grammatical reward function including label hierarchy, attribute hierarchy, and structural hierarchy to evaluate the generation quality of the preset model. It verifies the label matching, attribute usage, and table structure of Hypertext Markup Language (HTML) respectively, and then combines reinforcement learning methods to adjust the preset model to obtain the target model. This effectively solves the grammatical errors of label mismatch, invalid attribute values, and incomplete structure output by the preset model, making the model conversion results usable. Embodiments of this application also provide a table image conversion device, which is used to convert a target table image using the target model to obtain HTML conversion results. The target model is obtained by training and adjusting the grammatical layer reward value calculated from the hierarchical grammatical reward function on the hypertext markup language sequence. The hypertext markup language sequence is obtained by converting the sample table image using the preset model before adjustment. The table image conversion method and apparatus provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for converting table images, characterized in that, Includes the following steps: The target table image is converted using a target model to obtain the Hypertext Markup Language conversion result; The target model is a model obtained by training and adjusting the grammatical layer reward value calculated on the hypertext markup language sequence based on the hierarchical grammatical reward function. The hypertext markup language sequence is obtained by converting the sample table image using the preset model before adjustment.

2. The table image conversion method according to claim 1, characterized in that, The hierarchical grammar reward function includes a label hierarchy, an attribute hierarchy, and a structural hierarchy; The grammar layer reward value is obtained by weighted summation of the tag pairing correctness score, the attribute usage normativity score, and the table structure integrity score, wherein the tag pairing correctness score is the quality assessment result of the hypertext markup language sequence at the tag level, the attribute usage normativity score is the quality assessment result of the hypertext markup language sequence at the attribute level, and the table structure integrity score is the quality assessment result of the hypertext markup language sequence at the structure level.

3. The table image conversion method according to claim 2, characterized in that, The generation quality assessment of the hypertext markup language sequence based on the tag hierarchy includes: Traverse the hypertext markup language sequence, extract all tags and verify them, and record the number of tag matching errors; The tag matching accuracy score is obtained based on the number of tag matching errors and the total number of tags.

4. The table image conversion method according to claim 3, characterized in that, The step of extracting and verifying all tags includes: Push the start label onto the stack and verify the match between the end label and the start label at the top of the stack. If there is no match or the stack is empty, record the label pairing error. Verify the nesting rules of the tags, and record tag pairing errors if the preset nesting rules are violated.

5. The table image conversion method according to claim 3, characterized in that, The generation quality assessment of the hypertext markup language sequence based on the attribute hierarchy includes: Perform attribute validation on the attributes of the tag and obtain the attribute validation results; Based on the attribute verification results, a weighted calculation is performed according to the preset attribute value weights to obtain the attribute usage standardization score.

6. The table image conversion method according to claim 5, characterized in that, The attribute validation includes attribute name validity validation, attribute value validity validation, and attribute combination rationality validation.

7. The table image conversion method according to claim 2, characterized in that, The generation quality assessment of the hypertext markup language sequence based on the aforementioned structural hierarchy includes: A virtual grid matrix is ​​constructed to record the cell occupancy at each position in the hypertext markup language sequence. Matrix integrity is verified based on the virtual grid matrix to obtain a matrix integrity score. Verify whether there are any conflicts in the merged cells, and obtain a merge validity score based on the number of conflicting cells and the total number of cells; Perform a content integrity check on the content of each cell and obtain a content integrity score; The table structure integrity score is obtained based on the matrix integrity score, the merge validity score, and the content integrity score.

8. The table image conversion method according to claim 1, characterized in that, The table image conversion method also includes: For the same sample table image, G hypertext markup language sequences are generated independently using the preset model; where G is a positive integer.

9. The table image conversion method according to claim 8, characterized in that, The table image conversion method also includes: Based on the grammar layer reward value, the average reward of all sequences and the standard deviation of the rewards of all sequences are obtained to obtain the relative advantage value; Based on the relative advantage value, the target optimization value of the reinforcement learning method is obtained, and the preset model is adjusted using the reinforcement learning method to obtain the target model.

10. A table image conversion device, characterized in that, This device is used to convert a target table image using a target model to obtain a Hypertext Markup Language conversion result; The target model is a model obtained by training and adjusting the grammatical layer reward value calculated on the hypertext markup language sequence based on the hierarchical grammatical reward function. The hypertext markup language sequence is obtained by converting the sample table image using the preset model before adjustment.

Images