Training sample synthesis method and device, storage medium and program product

Abstract

The invention provides a training sample synthesis method and device, a storage medium and a program product. The training sample synthesis method comprises the following steps: acquiring a reference editable table comprising cell position information and original text contents in cells; performing semantic analysis on the original text content, and determining a semantic constraint relationship of each data column; on the basis of the semantic constraint relation, generating replacement text content and a corresponding content verification label; replacing the original text content with the replacement text content, generating a first-class editable table, rendering the first-class editable table into a first-class table image, and generating a table line position label for the first-class table image based on the cell position information of the reference editable table; and taking the first type of table images and the corresponding table line position labels as training samples of a table line segmentation model, and taking the first type of editable tables and the corresponding content verification labels as training samples of a table content verification model.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of data synthesis, and in particular to a training sample synthesis method, an electronic device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] In the process of informatization and digitization, a large number of paper documents or scanned images carrying key information need to be converted into structured data that can be calculated and analyzed. This process is particularly important in many fields that involve standardized data collection and processing, including a large number of fixed-format, logically rigorous forms such as various reporting forms, statistical forms, and data lists, which need to be converted into computer-editable forms.

[0003] The traditional method relies on manual recognition of form images and manual entry of information. This process is not only time-consuming and labor-intensive, but also prone to human error, becoming a bottleneck in data flow.

[0004] To improve efficiency, automatic form recognition technology based on artificial intelligence is a further solution. However, the training and optimization of such data-driven models rely on a large amount of labeled form data that is highly relevant to the target application scenario.

[0005] In practical applications, especially in the early stages of a project or for specific format forms, such high-quality labeled data is often very scarce and difficult to obtain directly to meet the needs of large-scale model training. The lack of such high-quality training samples greatly restricts the improvement of the performance of related models involved in the form recognition process and their effective application in practical scenarios. SUMMARY

[0006] Therefore, one or more embodiments of the present specification provide technical solutions as follows: According to a first aspect of one or more embodiments of the present specification, a training sample synthesis method is provided, comprising: obtaining a reference editable form, the reference editable form including cell position information and original text content in the cells; performing semantic analysis on the original text content to determine semantic constraint relationships of each data column in the reference editable form; generating replacement text content and corresponding content verification labels based on the semantic constraint relationships, the content verification labels being used to indicate whether the replacement text content conforms to the semantic constraint relationships; replacing the original text content with the replacement text content to generate a first type of editable form, rendering the first type of editable form into a first type of form image, and generating form line position labels for the first type of form image based on the cell position information of the reference editable form; The first type of table image and its corresponding table line position labels are used as training samples for the table line segmentation model, and the first type of editable table and its corresponding content verification labels are used as training samples for the table content verification model.

[0007] According to a second aspect of the embodiments of this specification, an electronic device is provided, comprising: processor; Memory used to store processor-executable instructions; Wherein, when the processor executes the executable instructions, it is used to implement the method described in the first aspect.

[0008] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] As described in the above embodiments, this specification obtains a reference editable table containing cell position information and original text content, performs semantic analysis on the original text content to determine the semantic constraints of each data column, and generates replacement text content and corresponding content verification labels accordingly. Then, the original text content is replaced with the replacement text content to generate a first type of editable table and rendered as a first type of table image. Table line position labels are generated by combining the cell position information of the reference table, ultimately obtaining training samples for the table line segmentation model and the table content verification model. By synchronously generating training samples suitable for two different tasks—the table line segmentation model and the table content verification model—through a unified process, this collaborative generation mechanism avoids the repetitive work of preparing data separately for different models, significantly improving the efficiency and consistency of data production. Furthermore, by automatically generating supervision labels (such as table line position labels and content verification labels), the reliance on manual annotation is greatly reduced, solving the pain points of difficulty and high cost in obtaining high-quality labeled data, and providing reliable data support for the rapid development and iteration of the table line segmentation model and the table content verification model.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0012] Figure 1 This is a flowchart of a training sample synthesis method provided in an exemplary embodiment.

[0013] Figure 2This is a schematic diagram of obtaining a first type of editable table provided in an exemplary embodiment.

[0014] Figure 3 This is a schematic diagram of obtaining a second type of editable table provided in an exemplary embodiment.

[0015] Figure 4 This is a schematic diagram of obtaining a third type of editable table provided in an exemplary embodiment.

[0016] Figure 5 This is a schematic diagram of the structure of a device provided in an exemplary embodiment. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0018] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0019] In the field of table recognition technology, the performance of deep learning is highly dependent on the scale and quality of the training data. However, in practical applications, especially in scenarios such as finance and government affairs where the accuracy and logical consistency of data are extremely important, acquiring large amounts of structurally complex, compliant, and precisely labeled table data is costly and cumbersome, constituting a major bottleneck in model development. Therefore, this specification provides a training sample synthesis method that uses an intelligent data generation strategy to construct large-scale, high-quality table datasets at low cost.

[0020] The training sample synthesis method can be performed by electronic devices, including but not limited to servers (single-machine servers, cluster servers, cloud servers), desktop computers, portable computers, smart terminals (smartphones, tablets), and edge computing devices.

[0021] Please see Figure 1 The flowchart of a training sample synthesis method is shown, which includes: In S100, a reference editable table is retrieved, which includes cell location information and the original text content in the cells.

[0022] For example, the format of a reference editable table can be any electronic format that can represent the table structure and text content. This includes, but is not limited to, structured data files such as HTML, JSON, or XML. The electronic device reads the cell structure information and raw text content by calling the appropriate parsing library.

[0023] In one possible implementation, the electronic device can receive or read reference editable table data through a preset input interface. These interfaces may include local storage reading (such as accessing table files in the device's hard drive or flash memory), network data receiving (such as obtaining table data transmitted from a remote server via a communication module), or importing from external devices (such as obtaining tables by connecting to other devices via USB or Bluetooth).

[0024] In another possible implementation, the electronic device can acquire a reference table image related to the target application scenario, wherein the reference table image is a wired table.

[0025] In the target application scenario, tables are presented in various forms, including wired tables with clear row and column boundaries, wireless tables whose structure is only indicated by spacing or content partitioning, and incomplete tables with missing horizontal or vertical lines. For wired tables, the image taken can be directly used as a reference table image because it contains a complete table line structure without additional processing. However, images of wireless or incomplete tables lack clear line boundaries and cannot directly provide effective recognition targets for subsequent table line segmentation models. The core function of table line segmentation models is to identify and locate line segment features in images. If there are no complete lines or the lines are missing, the model will not be able to accurately capture the row and column division logic of the table, leading to deviations in subsequent steps such as cell position calculation and text-cell association. Therefore, pre-defined annotation software can be used to add complete horizontal and vertical line positions to the image according to the row and column layout that such tables should have, restoring the structural boundaries that the table should have. This transforms the annotated image into a wired table style, ensuring the integrity of the table's structural information, and then uses it as a reference table image for subsequent processes.

[0026] The electronic device then uses a table line segmentation model to segment the reference table image, obtaining the position information of the table lines in the reference table image; the table lines include horizontal and vertical lines. It should be noted that this table line segmentation model is the model that needs to be continuously optimized through sample training. At the current stage, it may have only been trained with a small number of samples, or even be in an initial state without training. Its segmentation accuracy has not yet reached its optimal level, but it can meet the initial structure extraction requirements.

[0027] For example, a table line segmentation model can adopt an encoder-decoder architecture: the encoder extracts deep features such as line texture and edges of the table image through multi-layer convolution and pooling downsampling; the decoder upsamples to restore the image resolution and retains fine-grained details by combining skip connections; the model input is a reference table image (grayscale / RGB format), and the output is pixel-level table line position information to accurately locate the position of horizontal and vertical lines.

[0028] This embodiment does not impose any restrictions on the format for representing table line position information; specific settings can be made according to the actual application scenario. For example, the format for representing table line position information includes, but is not limited to: (1) The position information of the table lines can be represented by a grayscale image, where the pixel value of the horizontal line is 1, the pixel value of the vertical line is 2, and the pixel value of the background is 0.

[0029] (2) The position information of the table lines can be in the form of a coordinate list, which records the pixel coordinates of the start and end points of each table line in a two-dimensional array. For example, horizontal lines are represented by (x_start,y,x_end,y), where y is the fixed vertical coordinate of the horizontal line, and x_start and x_end are the horizontal coordinates of the left and right endpoints of the horizontal line. Vertical lines are represented by (x,y_start,x,y_end), where x is the fixed horizontal coordinate of the vertical line, and y_start and y_end are the vertical coordinates of the upper and lower endpoints of the vertical line.

[0030] (3) Table line position information can be recorded using the standard data structure of vector graphics, including parameters such as line length, direction, position coordinates, and line width.

[0031] (4) Dual mask image format: Generate two independent image files, a horizontal line mask image and a vertical line mask image. In the horizontal line mask image, only the horizontal lines of the table are retained, the pixel value of the line area is 255 (foreground), and the other areas are 0 (background). Similarly, the vertical line mask image only retains the vertical lines. By separating the two images, the two types of lines can be clearly distinguished, which is suitable for scenarios where the line type is processed separately.

[0032] For example, since the current table line segmentation model may be in its initial stage, the output table line position information may contain deviations, such as broken lines, local burrs, positional offsets, or incomplete line recognition due to image blurring. These inaccurate position information will directly affect the accuracy of subsequent intersection point calculations, leading to incorrect cell position positioning. Therefore, further correction processing of the table line position information is needed to ensure data accuracy.

[0033] One type of correction method is morphological operation correction, which uses morphological operators (such as dilation and erosion operators) in image processing algorithms to perform line segment fusion. Here, the operator parameters need to be set to a small size. This is because if the operator is too large, the table lines in different rows and columns will stick together, destroying the original structure of the table. Small-sized operators can accurately repair minor line breaks and remove local burrs, while preserving the original position and shape of the table lines to the greatest extent, thus achieving fine correction.

[0034] Another correction method is manual correction. For serious errors that cannot be corrected by morphological operations, such as large areas of missing lines, obvious positional offsets, and misidentification of lines, the positional information of table lines can be adjusted through manual review to ensure that the position and length of each line perfectly match the actual structure of the table. It is important to note that the execution of correction is phased: in the early stages of model training, due to the low segmentation accuracy, the correction step is indispensable; as training progresses, the recognition accuracy of the table line segmentation model gradually improves, and when the segmentation results meet the accuracy requirements of subsequent steps, the correction process can be omitted to improve the overall efficiency of the process.

[0035] Subsequently, after obtaining accurate (or corrected) table line position information, the electronic device will deduce the intersection point position information between horizontal and vertical lines through coordinate calculations. Specifically, the electronic device will iterate through the position information of all horizontal and vertical lines, and for each horizontal and vertical line, calculate the intersection coordinates, which are the intersection points of the table lines. All intersection points will be sorted and stored according to the row and column order of the table, forming a complete set of intersection points.

[0036] Based on this intersection point information, the cell's positional boundaries can be clearly defined: four adjacent intersection points (corresponding to the top left, top right, bottom left, and bottom right corners, respectively) constitute the four vertices of a cell. By recording the pixel coordinates of these four vertices, the spatial position range of each cell in the reference table image can be accurately determined, including the cell's width, height, and overall layout within the image. This process transforms the linear positional information of table lines into the regional positional information of cells, providing a spatial basis for the subsequent precise matching of text content with cells.

[0037] Furthermore, the electronic device can activate the optical character recognition (OCR) engine to perform a comprehensive scan and character recognition of the reference table image. During this process, the OCR engine analyzes the text content in the image region by region, converting the visualized characters (including numbers, letters, and Chinese characters) into editable text data, i.e., the original text content. Simultaneously, the OCR engine records the specific pixel location range of each piece of original text in the image, forming the original text content location information—usually represented by the top-left and bottom-right pixel coordinates of the area where the text is located—ensuring that each piece of text has a unique spatial location identifier. This achieves the conversion of image text into structured text, making it text data that can be further processed by the electronic device. The synchronously extracted text location information provides a matching basis for associating text content with cells, ensuring that subsequent determination of "which cell a certain piece of text belongs to" can be accurately made, avoiding mismatches between text and cells.

[0038] Finally, the electronic device establishes a matching relationship between the original text content's location information and the cell's location information. The device determines whether the location coordinates of each piece of original text content completely or primarily fall within the location coordinates of a certain cell, for example, by whether the center coordinates of the text content are within the cell boundary, or whether the overlap between the text area and the cell area exceeds a preset threshold. If the original text content completely or primarily falls within the location coordinates of a certain cell, then the original text content is bound to the corresponding cell, ensuring that each cell can be associated with its corresponding text information.

[0039] After associating all text content with cells, the electronic device integrates cell location information and associated original text content to generate a reference editable table. This table retains the row and column structure and cell layout of the original reference table image, while also possessing editable attributes. Its data is stored in a structured format and can be directly used for subsequent steps such as semantic analysis and text content replacement generation.

[0040] In this embodiment, a series of standardized operations such as annotation, segmentation, correction, recognition, and association are used to ensure the reliability of the final generated reference editable table related to the target application scenario in terms of structural accuracy and content integrity, providing solid support for the high-quality synthesis of subsequent training samples.

[0041] In S102, semantic analysis is performed on the original text content to determine the semantic constraints of each data column in the reference editable table.

[0042] In this step, the electronic device can use natural language processing algorithms to perform batch analysis of the original text content. First, the electronic device can group the original text content by data columns. For all the text in each column, it will perform operations such as part-of-speech tagging, data type judgment (e.g., distinguishing between numbers, dates, text, emails, etc.), and format feature extraction (e.g., whether it contains specific symbols, character length range) to uncover the common features of the data within the column. Based on this, the electronic device will further summarize the constraints of the data in that column, including value range (e.g., dates must be within a reasonable time interval), format specifications (e.g., emails must contain the @ symbol), and logical relationships with other data columns (e.g., data in one column must have a numerical relationship with data in another column), ultimately forming the semantic constraint relationships between the data columns.

[0043] Semantic constraints are an inherent rule system for tabular data, used to regulate the content logic within and between data columns, clarify the reasonable boundaries and relationships of the data, and provide an objective basis for subsequent generation of replacement text content and determination of content verification tags. This ensures that training samples can accurately cover various scenarios of data compliance and non-compliance. For example, semantic constraints include intra-column constraints and / or semantic relationships between different data columns.

[0044] Intra-column constraints are uniform rules that all data within the same column must follow. They ensure the consistency, standardization, and validity of data within a single column, preventing logical inconsistencies or invalid values. For example, intra-column constraints include at least one of the following: data type matching and value range restrictions.

[0045] Data type matching refers to all original text content in the same data column belonging to the same preset data type. The classification of data types must conform to the actual data characteristics, including but not limited to date, numeric, text, and specific format types. For example, if the original text content of a data column is in the format of 20230512, 2024-08-20, etc., electronic devices can determine that the data type of this column is date through semantic analysis, and the constraint is that all data in the column must conform to the date format specification. In addition, there are specific format types. If all data in a column is in the form of "xxx@xx.com", it is determined to be an email address, and the constraint is that all data in the column must contain the @ symbol and the domain name suffix. If all data is 11 digits, it is determined to be a mobile phone number, and the constraint is that all data in the column must be 11 pure numbers.

[0046] Value range restrictions refer to the requirement that, based on a clearly defined data type, the original text content of the same data column must fall within specific value boundaries or belong to a preset enumeration set, with no values ​​exceeding a reasonable range. This constraint is divided into two categories: numerical range restrictions and enumeration value restrictions. Numerical range restrictions apply to numerical data columns. For example, if a column contains student exam scores and the original text content is all between 0 and 100, the electronic device will determine its constraint as a value range of 0-100. Enumeration value restrictions apply to columns where the data values ​​have a fixed set of possible choices. For example, if a column contains gender and the original text content is only in three categories: male, female, and other, the constraint would be: enumeration values: male, female, other.

[0047] Semantic relationships between different data columns are logical, computational, or mapping-level association rules that exist between them. Their core objective is to ensure the synergy, consistency, and logical self-consistency of multiple columns, reflecting the overall interconnectedness of the tabular data, rather than the independent rules of a single column. For example, semantic relationships between different data columns include at least one of the following: mathematical calculation relationships, logical dependencies, and dictionary mapping relationships.

[0048] Mathematical calculation relationships refer to the ability of one or more columns of data (called source data columns) to precisely obtain another column of data (called target data columns) through specific mathematical operations (such as addition, subtraction, multiplication, division, summation, averaging, etc.). This is the core manifestation of quantitative relationships between data columns. For example, if a table contains three columns: unit price, quantity, and total price, an electronic device can analyze the original text content and find that: Total price = Unit price × Quantity, thus determining that a mathematical calculation relationship exists between the three columns. See another example. Figure 2 Each value in the total score column is equal to the sum of the values ​​in the corresponding row's math, English, and Chinese columns.

[0049] Logical dependency refers to the relationship where the validity, reasonableness, or specific content of a data column (called the dependent column) depends on the value of another data column (called the reference column). It is the core manifestation of qualitative relationships between data columns. For example, in a table with two columns (order status reference column and delivery date reference column), if the order status column is "completed," then the delivery date column must have a specific date that is later than the shipping date column; if the order status column is "not shipped," then the delivery date column must be empty. This demonstrates the constraint that the value of the reference column imposes on the value of the dependent column.

[0050] Dictionary mapping refers to a unique or fixed correspondence between each value of a column of data (called the key column) and a fixed set of values ​​in another column of data (called the value column). Similar to the key-value mapping logic in a dictionary, it represents a fixed association between data columns. For example, consider two columns: Product Category (key column) and Product Subcategory (value column). If the key column value is "electronic products," the value column values ​​could be a fixed set such as "mobile phones," "computers," and "tablets." Similarly, the association between Department Number (key column) and Department Name (value column) also falls into this category. If the key column value is 001, the value column value is "Human Resources Department"; if the key column value is 002, the value column value is "Technology Department," forming a unique mapping.

[0051] In S104, replacement text content and corresponding content verification tags are generated based on semantic constraint relationships. The content verification tags are used to indicate whether the replacement text content conforms to semantic constraint relationships.

[0052] In this step, the electronic device can automatically generate replacement text content for at least one cell and corresponding content verification tags based on semantic constraints, reducing the cost of manual generation and annotation and avoiding potential oversights during manual operation.

[0053] For example, please refer to Figure 2 Electronic devices can generate replacement text content that conforms to semantic constraints and configure content verification labels indicating positive samples for the replacement text content. This enables the construction of high-quality positive training samples—samples that accurately represent the content characteristics of table data under compliant conditions, providing the table content verification model with a learning basis for "what constitutes reasonable data" and helping the model establish a positive understanding of semantic constraints. The generation process strictly follows the intra-column constraints and inter-column semantic relationships established earlier, employing targeted generation strategies for different constraint types to ensure that the replacement content not only conforms to the rules but also possesses diversity and authenticity.

[0054] Generating replacement text content that conforms to semantic constraints includes, but is not limited to, at least one of the following operations: ① In columns with mathematical calculation relationships, the electronic device generates replacement text content that satisfies the mathematical calculation relationship according to the calculation formula indicated by the mathematical calculation relationship. For example, the electronic device first defines the source data column, target data column, and specific calculation formula in the mathematical calculation relationship, such as total price = unit price × quantity. During generation, a two-way strategy is adopted to derive the target column data from the source column data or to derive the source column data from the target column data to ensure a closed loop of calculation logic. For example, for the calculation relationship of unit price (source column 1), quantity (source column 2), and total price (target column), the electronic device can first randomly generate replacement data for the source columns that conform to the value range of the unit price column (e.g., 10-100 yuan) and the value range of the quantity column (e.g., 1-10 pieces) (e.g., unit price 35 yuan, quantity 6 pieces), and then calculate the replacement data for the target column of 210 yuan using the formula "35×6"; or it can first determine the target column data (e.g., total price 180 yuan), and then randomly specify one of the source column data (e.g., quantity 5 pieces), and obtain the other source column data of 36 yuan by reverse derivation formula "180÷5".

[0055] ② In columns with logical dependencies, generate replacement text content that conforms to logical consistency. For example, the electronic device first clarifies the baseline and dependent columns in the logical dependency relationship, as well as their logical correspondence rules. For instance, if the baseline column value is A, the dependent column must meet condition B. During generation, the baseline column data is used as the core, and compliant data for the dependent column is generated in conjunction, ensuring logical consistency between the values ​​of the two columns. For example, regarding the relationship between order status (baseline column) and delivery date (dependent column), if the baseline column generates "Completed," then the dependent column generates a reasonable date later than the shipping date (another baseline column), such as a shipping date of 2024-05-10 and a delivery date of 2024-05-12; if the baseline column generates "Not Shipped," then the dependent column remains empty.

[0056] ③ In columns with dictionary mapping relationships, generate valid values ​​corresponding to the dictionary indicated by the dictionary mapping relationship. For example, the electronic device first extracts the "key-value" correspondence set in the dictionary mapping relationship, such as province-province capital, department number-department name, product category-product subcategory. When generating, it uses the key column data as an index to match the corresponding valid values ​​from the preset mapping dictionary as the replacement content of the value column, ensuring that the mapping relationship between the key column and the value column is accurate.

[0057] ④ In columns with data type matching relationships, generate replacement content that conforms to the predefined data type of that column. For example, for date columns, generate dates that conform to preset formats such as "YYYY-MM-DD", "YYYY / MM / DD", and "YYYYMMDD" to avoid format or logical errors such as "2024-13-01" (month out of range) and "2024 / 06 / 31" (date out of range).

[0058] ⑤ In columns with value range restrictions, generate replacement content within the preset value range. For example, for columns with numerical range restrictions, such as student scores of 0-100, the electronic device will randomly generate integers or values ​​with one decimal place within the range of 0-100, such as 85, 92.5, and 63, to avoid negative numbers, values ​​greater than 100, or non-numeric content.

[0059] The above generation operations are all based on semantic constraints. Through targeted generation strategies, they ensure that the replacement text content fully complies with the rules in terms of format, value, and logical connections. Simultaneously, the electronic device can control the diversity of data during the generation process (such as different values, different texts, and different mapping combinations) to avoid the homogenization of positive samples. This allows the table content verification model to comprehensively learn the characteristic patterns of compliant data through such samples, providing a solid positive reference for the subsequent accurate identification of non-compliant data. After generating compliant replacement text content, the electronic device will configure a unified "positive sample" content verification label for each replacement text content, such as in the format of 0 or True, clearly indicating that the content conforms to the semantic constraints, providing clear supervision signals for model training.

[0060] For example, electronic devices can generate replacement text content that violates semantic constraints and configure content verification labels indicating negative samples for the replacement text content. This enables the construction of reverse training samples covering various violation scenarios—these samples clearly present the typical characteristics of non-compliant tabular data, complementing the positive samples and helping the tabular content verification model establish a complete judgment boundary between "compliance and violation," accurately learning the core logic for identifying data violations of constraint rules. The generation process focuses on disrupting the previously established intra-column constraints and inter-column semantic relationships, designing targeted violation generation strategies for different constraint types to ensure clear violation points and realistic scenarios.

[0061] Generating replacement text content that violates semantic constraints includes, but is not limited to, at least one of the following operations: ① In a column with mathematical calculation relationships, modify any data item to invalidate the mathematical calculation relationship; for example, you can modify the value of any source data column. Please refer to [link to relevant documentation]. Figure 2 For example, regarding the relationship "total score = math + English + Chinese", changing the total score from 245 to 249 while keeping math, English, and Chinese unchanged leads to a calculation contradiction of "70 + 85 + 90 ≠ 249".

[0062] ② In columns with logical dependencies, modify any data item to make the logical dependency invalid; for example, for the relationship of "whether married - spouse's name", change the base column "whether married" from "no" to "yes", but the dependent column "spouse's name" still remains empty, forming a logical contradiction of "married but without spouse's name".

[0063] ③ In columns with dictionary mapping relationships, generate invalid values ​​that do not conform to the dictionary mapping relationship indication; for example, for a one-key-multiple-value mapping relationship, such as product category-product subcategory, generate content that does not belong to the corresponding category subcategory set. For example, when the key column is electronic products, the value column generates towels (which belong to the household goods subcategory).

[0064] ④ In columns with data type matching relationships, replace with invalid content that does not conform to the predefined data type of the column; for example, for a mobile phone number column, generate content that is not 11 digits, contains mixed letters and symbols, or violates the number segment rules, such as "138xxxx567" (10 digits) and "159xxxyy9012" (mixed with letters).

[0065] ⑤ In columns with value range restrictions, generate invalid content that exceeds the preset value range; for example, for columns with numerical range restrictions, generate values ​​that are lower than the minimum value or higher than the maximum value, such as generating 120 points and -5 points in the "Student Score (0-100 points)" column.

[0066] Electronic devices can employ diverse violation generation strategies to cover various violation scenarios that may occur in real-world situations, avoiding the homogenization of negative sample scenarios. After generating replacement text content that violates constraints, the electronic device will configure a unified "negative sample" content verification label for each piece of content, such as in the format of 1 or False, clearly indicating that the content violates semantic constraints. This clearly distinguishes it from positive sample labels, providing clear reverse supervision signals for model training and helping the model build a comprehensive and accurate compliance judgment system.

[0067] In S106, the original text content is replaced with replacement text content to generate a first type of editable table, and the first type of editable table is rendered as a first type of table image. Table line position labels are generated for the first type of table image based on the cell position information of the reference editable table.

[0068] In this step, please refer to Figure 2 The electronic device replaces the original text content of at least one cell in the editable table with the replacement text content generated in S104, forming a complete first-type editable table. This table retains the row and column structure, cell layout and other attributes of the original table, only updates the text content, and is still stored in the form of structured data.

[0069] Furthermore, the electronic device activates the image rendering engine to restore the visual presentation of the table based on the structured data of the first type of editable table, including the style of the table lines, the font and position of the text, the size and spacing of the cells, etc., and converts the structured table into a pixel-level first type of table image.

[0070] Meanwhile, based on the existing cell position information of the editable table, the device deduces the specific position of the table lines through coordinate calculation: horizontal table lines correspond to the upper and lower boundary coordinates of each cell, and vertical table lines correspond to the left and right boundary coordinates of each cell. The device organizes the start coordinates, end coordinates and other information of these lines into table line position labels and establishes a unique association with the table image.

[0071] In one possible implementation, the first type of editable table is saved as an HTML file. The electronic device can load and render the HTML file using a browser. The browser's parsing engine analyzes the HTML tags and embedded basic style information (such as table border width, cell spacing, and text font and size), transforming the structured table data into a visual table according to web page rendering standards. Simultaneously, this visual table is fully displayed in the browser window's content area, ensuring that the table's row and column boundaries, text layout, table line styles, and other visual features are consistent with the table presentation in the actual application scenario. After the browser completes rendering and stably displays the table, the electronic device can activate the system-level screenshot module to perform a precise screenshot of the corresponding display area of ​​the table in the browser window. This embodiment leverages the browser's mature HTML rendering capabilities, achieving accurate reproduction of the table's visual effect without the need for additional complex rendering engine development. Simultaneously, the universality of the HTML format ensures consistency in rendering results across different devices and browser environments. The screenshot operation efficiently completes the conversion from "visual interface to image data." The entire process balances efficiency and accuracy, providing a reliable guarantee for the standardized generation of training samples.

[0072] In another possible implementation, the electronic device can also draw table elements directly on the image canvas based on the structured data of the first type of editable table by calling a professional image processing library at the programming level. After all elements are drawn, the canvas is saved as an image file in a specified format to form a first type of table image.

[0073] In S108, the first type of table image and the corresponding table line position label are used as training samples for the table line segmentation model, and the first type of editable table and the corresponding content verification label are used as training samples for the table content verification model.

[0074] For example, the table line segmentation model takes a first-class table image as input data and table line position labels as supervision targets to train the model to accurately identify and segment table lines from the image; the table content verification model takes a first-class editable table as input data and content verification labels as supervision targets to train the model to determine whether the table content conforms to the preset semantic constraints.

[0075] For example, the table content validation model is built on the Large Language Model (LLM) and adds a table structure awareness and adaptation layer: the input is a first type of editable table, which is first converted into a text sequence with row and column position identifiers by the table structure awareness and adaptation layer; the core LLM layer parses the text sequence and matches it with preset semantic constraint relationships; the output is the judgment result (Boolean value / compliance probability value) of the corresponding content validation tag, indicating whether the replacement text content conforms to the semantic constraints.

[0076] For example, the training process of the table content verification model effectively improves model performance through the collaborative support of positive and negative samples. Positive samples (replacement content that conforms to semantic constraints) provide rich compliance examples, helping the model learn core constraint features (such as format, value, and logical connections), build compliance awareness, and solve the problem of limited compliance cases in real samples. Negative samples (content that violates constraints) fill the gap of scarce violation cases in real scenarios, clarifying the specific characteristics of non-compliant situations. The two form a clear contrast, enabling the model to clearly distinguish between compliance and violation features, strengthen the judgment boundary, avoid misjudgment and omission, fundamentally improve the defect of ambiguous feature boundaries in real samples, and significantly improve the model's judgment accuracy, generalization ability, and anti-interference ability.

[0077] The training sample synthesis method provided in this embodiment generates training samples suitable for two different tasks—table line segmentation and table content validation—through a unified process. This collaborative generation mechanism avoids the repetitive work of preparing data separately for different models, significantly improving the efficiency and consistency of data production. Furthermore, by automatically generating supervision labels (such as table line position labels and content validation labels), it greatly reduces the reliance on manual annotation, addressing the pain points of difficulty and high cost in obtaining high-quality labeled data, and providing reliable data support for the rapid development and iteration of table line segmentation and table content validation models.

[0078] In some embodiments, in order to further enrich the structural diversity of the training samples of the table line segmentation model, the electronic device may also perform random cell structure modification operations on the cells in the reference editable table to generate a second type of editable table; wherein, the random cell structure modification operations include: merging any adjacent cells and / or splitting any cells.

[0079] For merge operations, please refer to Figure 3The electronic device randomly selects any adjacent cells in the table for merging. Adjacency includes horizontal adjacency (consecutive cells in the same row) and vertical adjacency (consecutive cells in the same column). Multiple groups of adjacent cells can also be selected to form a larger area for merging, such as merging a 2×2 cell matrix into a single cell. During the merging process, the device automatically retains the corresponding replacement text content in the merged cells. For example, if multiple cells are being merged, the text can be concatenated in order or the core content can be retained. At the same time, the internal table lines within the merged area are deleted, leaving only the outer boundary of the merged cell to ensure the integrity of the table content after structural modification.

[0080] For the splitting operation, the electronic device randomly selects any cell in the table for splitting. The splitting method can be flexibly adjusted: a single cell can be split horizontally into two or more cells (top and bottom), or vertically into two or more cells (left and right). The size of the split sub-cells is evenly distributed according to the original cell size or randomly and reasonably distributed. During the splitting process, the device assigns corresponding text content to each split sub-cell, such as semantically splitting or copying and filling the original cell text, and adding internal table lines within the split area to ensure that the boundaries of the sub-cells are clearly distinguishable.

[0081] It should be noted that the randomness of structural modification operations is reflected in the random selection of the operation object, operation type, and operation scope. Electronic devices can control the intensity of modification to avoid excessive merging or splitting that could completely distort the table structure. For example, merging all cells in an entire row into one ensures that the modified table still has recognizable row and column logic, which meets the learning requirements of the table line segmentation model.

[0082] After modifying the random cell structure, the electronic device integrates the modified cell structure information with the original replacement text content to generate a second type of editable table. The basic row and column framework of this table remains unchanged; only the combination of local cells is adjusted. Simultaneously, the device recalculates and updates the position information of each cell, including the overall boundary coordinates of the merged cells and / or the independent boundary coordinates of the split sub-cells, storing this information in structured data to ensure a complete match between the cell position information and the modified structure. This provides an accurate structural basis for subsequent image rendering and label generation.

[0083] Subsequently, the electronic device uses the same standardized rendering process as the one used to generate the first type of table image to render the second type of editable table into a second type of table image. Based on the cell position information of the second type of editable table, it generates table line position labels for the second type of table image. The label generation process needs to adapt to the table features after structural modification: for merged cell areas, the original internal table line position records are deleted, and only the external table line coordinates of the merged cells are retained; for split cell areas, the coordinates of the internal table lines generated by the split are added to ensure that the start and end positions of each new line are completely aligned with the boundaries of the sub-cells.

[0084] Finally, the electronic device uses the second type of table image and its corresponding table line position labels, along with the first type of table image and its corresponding table line position labels, as training samples for the table line segmentation model. The two types of samples complement each other: the first type of samples covers table scenarios with regular row and column structures, while the second type of samples covers complex structural scenarios involving cell merging and splitting, together constructing a comprehensive sample system.

[0085] This embodiment significantly broadens the learning scenario boundaries of the model, and the diverse training samples enhance the model's adaptability and segmentation accuracy in real-world scenarios. Simultaneously, the randomness of the structural modification operations ensures the diversity of the second type of samples, preventing the model from overfitting to specific merging or splitting patterns, further strengthening the model's generalization performance, and providing rich and high-quality data support for the comprehensive optimization of the table-line segmentation model.

[0086] In other embodiments, the electronic device performs a specified cell structure modification operation on a specific column in the reference editable table based on the semantic constraints of the data columns in the reference editable table. The specified cell structure modification operation includes at least one of the following: (1) Merge cells with semantic aggregation relationships. Semantic aggregation relationships refer to multiple adjacent cells whose content belongs to the same semantic category, shares a common superordinate concept, or belongs to the same group. Merging them can more clearly present the hierarchical relationship of the data. For example, please refer to Figure 4 For example, the Chinese and English columns can be merged into a single language subject column, and the sum of the values ​​in the corresponding rows of the Chinese and English columns can be calculated. Similarly, if multiple adjacent cells in a date column contain dates in March 2024, exhibiting time-based aggregation semantics, they can be merged into a single cell to reflect time-based grouping logic. During the merging process, the electronic device preserves the core semantic information of the merged cells, removes internal table lines within the merged area, and retains only the outer boundaries, ensuring that structural adjustments do not compromise semantic integrity.

[0087] (2) Splitting cells containing at least two semantic information items. The original content of these cells contains two or more independent, separable semantic units, and each semantic unit conforms to the semantic constraints of the corresponding column. Splitting makes the data presentation more accurate and standardized. For example, the content of a cell in the contact information column is 138xxxx5678|zhangsan@xxx.com, which contains two independent semantic information items: mobile phone number and email address. The two information items conform to the data type constraints of mobile phone number and email address, respectively. The electronic device will split this cell into two adjacent cells, which will respectively carry the mobile phone number and email address information. As another example, the content of a cell in the address column contains three semantic information items: province, city, and street. It can be split into three sub-cells for storage. During the splitting process, the electronic device will add internal table lines to divide the semantic units, ensuring that the boundaries of the sub-cells are clear and the semantic belonging of each sub-cell is clear.

[0088] After modifying the structure of a specific column, the electronic device modifies the original text content in the modified column according to semantic constraints. The adjustment strategy corresponds one-to-one with the structure modification type: for merged cells, the text content of the original multiple cells is integrated according to semantic aggregation rules; for split cells, the composite semantic content of the original cells is split into independent semantic units to ensure that each sub-cell carries only one semantic information and that the split content conforms to the semantic constraints of the corresponding column.

[0089] After the text content is adjusted, the electronic device generates a third type of editable table. Its structured data includes the modified cell structure information and the adjusted text content, while fully preserving the original semantic constraints. Subsequently, the electronic device, referring to the label configuration logic described earlier, configures content validation labels for the third type of editable table: if the adjusted text content conforms to the semantic constraints of the corresponding column, a positive sample label is configured; if there are violations of constraints, such as the split content not conforming to the data type or the aggregated content exceeding the value range, a negative sample label is configured. The content validation labels are associated one-to-one with the cell content, ensuring the accuracy and clarity of the annotations.

[0090] The electronic device employs a standardized rendering process consistent with that of the first and second types of table images to render the third type of editable table as a third type of table image. Simultaneously with generating the third type of table image, the electronic device derives and generates corresponding table line position labels based on the cell position information of the third type of editable table. The label generation process precisely adapts to the features after structural modifications: in merged areas, the original internal table line position records are deleted, retaining only the coordinate information of the external table lines; in split areas, coordinate records of internal table lines are added, ensuring that the start and end positions of the new lines are perfectly aligned with the boundaries of the sub-cells.

[0091] Finally, the electronic device can use the third type of table image and its corresponding table line position labels as training samples for the table line segmentation model, and the third type of editable table and its corresponding content verification labels as training samples for the table content verification model. This embodiment achieves synchronous upgrades of training samples for both models by using a reference editable table as a basis and driving structured modification and content adaptation through semantic constraints. This provides matching, high-quality training data for both the table line segmentation model and the table content verification model. This allows the table line segmentation model to access and learn to recognize various complex and unconventional table line arrangements, greatly improving the model's robustness and accuracy when encountering structurally variable tables in real-world scenarios. Furthermore, it provides the table content verification model with more diverse training data, training the model to understand deeper data logic beyond fixed layouts, making its verification capabilities more intelligent and powerful.

[0092] This data synergy ensures that when these two models are integrated into an end-to-end table recognition system, their understanding of this "structure-content" synergistic change develops synchronously. Combining a line segmentation model that can accurately identify merged cells with a content verification model that can understand the rationality of merged content will ultimately significantly improve the accuracy and reliability of the entire table recognition system.

[0093] In some embodiments, during the training process of a table line segmentation model, the electronic device can acquire training samples, including a first type of table image and its corresponding table line position labels, a second type of table image and its corresponding table line position labels, and a third type of table image and its corresponding table line position labels. The electronic device can input at least one of the first, second, and third type of table images into the table line segmentation model to be trained to obtain predicted table line information output by the model. Then, with the optimization objective of minimizing the error between the predicted table line information and the corresponding table line position labels, the parameters of the table line segmentation model are adjusted. Exemplarily, the table line segmentation model can be a convolutional neural network, a recurrent neural network, or a fully convolutional neural network with an encoder-decoder structure, etc., and this embodiment does not impose any limitations on this.

[0094] For example, when the table line segmentation model trained according to this scheme is put into practical application, one task is to efficiently and accurately convert tables contained in various paper documents or scanned images into computable and analyzable structured data. The specific application process is as follows: First, images of tables in paper documents are captured to obtain initial table images. For wired tables with complete table lines, the resulting images can be directly used for subsequent processing. For wired tables or tables with missing lines, image preprocessing is required to ensure the consistency and accuracy of the recognition process. This preprocessing is done by calling a preset annotation tool. Operators or automated programs accurately annotate the positions of complete horizontal and vertical lines in the image according to the logical layout of the table's rows and columns, thereby generating a table image with complete wireframe information that can be processed by the model.

[0095] Next, the processed table image is input into the trained table line segmentation model. The table line segmentation model infers from the table image and outputs table line position information, including the coordinates of all horizontal and vertical lines. Based on this table line position information, the electronic device further determines the boundary and position of each cell by calculating the coordinates of the intersection points between the horizontal and vertical lines. Simultaneously, the electronic device performs an optical character recognition process on the same table image to identify the content of each text block and its position coordinates in the image.

[0096] The electronic device then spatially matches the text location information with the previously calculated cell location information, associating each identified text element with its corresponding cell. In this way, the electronic device successfully constructs an editable table that combines physical structure and text content. This editable table can be saved as an HTML file to fully utilize its standard ability to describe table structures.

[0097] For specific needs regarding tables with or without lines, the visual presentation of the table can be precisely controlled by configuring the Cascading Style Sheets (CSS) properties associated with the HTML file. For tables with lines without lines, the display status of all table lines can be hidden by setting CSS properties, thus generating a final editable table with lines without lines. For tables with missing lines, specific table lines can be selectively hidden to accurately restore their original visual style, resulting in an editable table with missing lines. This process ensures accurate and efficient conversion from diverse paper forms to structured data.

[0098] In some embodiments, during the training process of the table content validation model, the electronic device can acquire training samples, including a first type of editable table and its corresponding content validation label, and a third type of editable table and its corresponding content validation label. The electronic device can input the first type of editable table and / or the third type of editable table into the table content validation model to be trained to obtain the predicted validation result output by the table content validation model. Then, with the optimization objective of minimizing the error between the predicted validation result and the corresponding content validation label, the parameters of the table content validation model are adjusted. For example, the table content validation model can be a pre-trained large language model based on Transformer, and this embodiment does not impose any limitations on it.

[0099] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0100] Figure 5 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 5 As shown, device 500 mainly consists of a communication interface 502, a user interface 504, a processor 506, and a data storage 508. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 510. The communication interface 502 enables device 500 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 502 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 502 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 502 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 502 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide-area wireless interfaces.

[0101] User interface 504 includes receiving user input and providing output to the user. Therefore, user interface 504 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 504 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 504 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 500 may support remote access from other devices via communication interface 502 or another physical interface (not shown). User interface 504 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 504 may also be configured as a display device for rendering or displaying text fragments.

[0102] Processor 506 may contain one or more general-purpose processors and / or special-purpose processors.

[0103] Data storage 508 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 506. Data storage 508 may include removable and non-removable components.

[0104] Processor 506 is capable of executing program instructions 518 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 508 to perform the various functions described herein. Data storage 508 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 500, enable device 500 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 518 by processor 506 may result in processor 506 using data 512.

[0105] For example, program instructions 518 may include an operating system 522 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 500 and one or more applications 520 (e.g., a browser, social application, or game application). Similarly, data 512 may include operating system data 516 and application data 514. Operating system data 516 is primarily accessible to the operating system 522, while application data 514 is primarily accessible to one or more applications 520. Application data 514 may reside in a file system visible or hidden from the user of device 500.

[0106] Application 520 can communicate with operating system 522 through one or more application programming interfaces (APIs). These APIs help application 520 read and / or write application data 514, transmit or receive information via communication interface 502, receive or display information on user interface 504, etc.

[0107] In some terminology, application 520 may be simply referred to as "app". Furthermore, application 520 can be downloaded to device 500 through one or more online app stores or app markets. However, applications can also be installed on device 500 in other ways, such as through a web browser or a physical interface on device 500 (e.g., a USB port).

[0108] In some embodiments, the training sample synthesis apparatus can be applied to, for example... Figure 5 The device shown is used to implement the technical solution of this specification. The training sample synthesis apparatus may include: The data acquisition module is used to acquire a reference editable table, which includes cell location information and the original text content in the cells.

[0109] The semantic analysis module is used to perform semantic analysis on the original text content and determine the semantic constraint relationships of each data column in the reference editable table.

[0110] The sample generation module is used to generate replacement text content and corresponding content verification tags based on the semantic constraint relationship. The content verification tags are used to indicate whether the replacement text content conforms to the semantic constraint relationship.

[0111] The sample generation module is further configured to replace the original text content with the replacement text content to generate a first type of editable table, render the first type of editable table as a first type of table image, and generate table line position labels for the first type of table image based on the cell position information of the reference editable table.

[0112] The sample generation module is further configured to use the first type of table image and the corresponding table line position labels as training samples for the table line segmentation model, and to use the first type of editable table and the corresponding content verification labels as training samples for the table content verification model.

[0113] In one implementation, the sample generation module is specifically used to generate replacement text content that conforms to the semantic constraint relationship, and configure a content verification label indicating a positive sample for the replacement text content; and / or, generate replacement text content that violates the semantic constraint relationship, and configure a content verification label indicating a negative sample for the replacement text content.

[0114] In one implementation, the semantic constraint relationship includes intra-column constraint relationship and / or semantic association relationship between different data columns; the intra-column constraint relationship includes at least one of the following: data type matching relationship and value range restriction; the semantic association relationship includes at least one of the following: mathematical calculation relationship, logical dependency relationship and dictionary mapping relationship.

[0115] The process of generating replacement text content that conforms to the semantic constraints includes at least one of the following operations: ① In columns with mathematical calculation relationships, generating replacement text content that satisfies the mathematical calculation relationships according to the calculation formulas indicated by the mathematical calculation relationships; ② In columns with logical dependency relationships, generating replacement text content that conforms to logical consistency; ③ In columns with dictionary mapping relationships, generating valid values ​​corresponding to the dictionaries indicated by the dictionary mapping relationships; ④ In columns with data type matching relationships, generating replacement content that conforms to the predefined data type of the column; ⑤ In columns with value range restrictions, generating replacement content that is within a preset value range. The generation of replacement text content that violates the semantic constraints includes at least one of the following operations: ① In a column with a mathematical calculation relationship, modify any data item to make the mathematical calculation relationship invalid; ② In a column with a logical dependency relationship, modify any data item to make the logical dependency relationship invalid; ③ In a column with a dictionary mapping relationship, generate invalid values ​​that do not conform to the dictionary mapping relationship; ④ In a column with a data type matching relationship, replace with invalid content that does not conform to the predefined data type of the column; ⑤ In a column with a value range restriction, generate invalid content that exceeds the preset value range.

[0116] In one implementation, the sample generation module is further configured to perform random cell structure modification operations on the cells in the reference editable table to generate a second type of editable table; wherein, the random cell structure modification operations include: merging any adjacent cells and / or splitting any cells; rendering the second type of editable table into a second type of table image, and generating table line position labels for the second type of table image based on the cell position information of the second type of editable table; and using the second type of table image and the corresponding table line position labels as training samples for the table line segmentation model.

[0117] In one implementation, the sample generation module is further configured to perform a specified cell structure modification operation on a specific column in the reference editable table based on the semantic constraint relationship of each data column in the reference editable table. The specified cell structure modification operation includes: merging cells with semantic aggregation relationship, and / or splitting cells containing at least two semantic information items; modifying the corresponding original text content in the modified specific column according to the semantic constraint relationship to generate a third type of editable table, and configuring content verification labels for the third type of editable table; rendering the third type of editable table into a third type of table image, and generating table line position labels for the third type of table image based on the cell position information of the third type of editable table; using the third type of table image and the corresponding table line position labels as training samples for a table line segmentation model, and using the third type of editable table and the corresponding content verification labels as training samples for a table content verification model.

[0118] In one implementation, the data acquisition module is specifically used to acquire a reference table image related to the target application scenario; segment the reference table image using the table line segmentation model to obtain table line position information in the reference table image; calculate the intersection position information between horizontal and vertical lines using the table line position information, and determine the cell position information in the reference table image based on the intersection position information; perform optical character recognition on the reference table image to obtain the original text content and the corresponding original text content position information; and associate the original text content with the corresponding cell based on the matching relationship between the cell position information and the original text content position information to generate a reference editable table corresponding to the reference table image.

[0119] In one implementation, the first type of editable table is saved as an HTML file.

[0120] The sample generation module is further configured to load and render the HTML format file using a browser, and display the rendering result in the browser window; and to perform a screenshot operation on the browser window to obtain the first type of table image.

[0121] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0122] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0123] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0124] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0125] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0126] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0127] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0128] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0129] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0130] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0131] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for synthesizing training samples, comprising: Obtain a reference editable table, which includes cell location information and the original text content in the cells; Perform semantic analysis on the original text content to determine the semantic constraint relationships of each data column in the reference editable table; Based on the semantic constraint relationship, a replacement text content and a corresponding content verification tag are generated. The content verification tag is used to indicate whether the replacement text content conforms to the semantic constraint relationship. The original text content is replaced with the replacement text content to generate a first type of editable table, and the first type of editable table is rendered into a first type of table image, and table line position labels are generated for the first type of table image based on the cell position information of the reference editable table. The first type of table image and its corresponding table line position labels are used as training samples for the table line segmentation model, and the first type of editable table and its corresponding content verification labels are used as training samples for the table content verification model.

2. The method according to claim 1, wherein generating replacement text content and corresponding content verification tags based on the semantic constraint relationship includes: Generate replacement text content that conforms to the semantic constraints, and configure content verification tags indicating positive samples for the replacement text content; And / or, Generate replacement text content that violates the semantic constraints, and configure content verification tags indicating negative samples for the replacement text content.

3. The method according to claim 2, wherein the semantic constraint relationship includes intra-column constraint relationship and / or semantic association relationship between different data columns; The intra-column constraints include at least one of the following: data type matching relationships and value range restrictions; The semantic association relationship includes at least one of the following: mathematical calculation relationship, logical dependency relationship, and dictionary mapping relationship; The generation of replacement text content that conforms to the semantic constraints includes at least one of the following operations: In columns with mathematical calculation relationships, replacement text content that satisfies the mathematical calculation relationship is generated according to the calculation formula indicated by the mathematical calculation relationship; In columns with logical dependencies, generate replacement text content that conforms to logical consistency; In columns with dictionary mapping relationships, generate valid values ​​corresponding to the dictionaries indicated by the dictionary mapping relationships; In columns with data type matching relationships, generate replacement content that matches the predefined data type of that column; In columns with value range restrictions, generate replacement content that falls within the preset value range; The generation of replacement text content that violates the semantic constraints includes at least one of the following operations: In a column with mathematical calculation relationships, modify any data item to make the mathematical calculation relationship invalid. In columns with logical dependencies, modify any data item to make the logical dependency invalid. In columns with dictionary mapping relationships, generate invalid values ​​for dictionaries that do not conform to the dictionary mapping relationship indication; In columns with data type matching relationships, replace with invalid content that does not conform to the predefined data type of that column; In columns with value range restrictions, generate invalid content that exceeds the preset value range.

4. The method according to claim 1, further comprising: Perform random cell structure modification operations on the cells in the reference editable table to generate a second type of editable table; wherein, the random cell structure modification operations include: merging any adjacent cells and / or splitting any cells; The second type of editable table is rendered into a second type of table image, and table line position labels are generated for the second type of table image based on the cell position information of the second type of editable table; The second type of table image and its corresponding table line position labels are used as training samples for the table line segmentation model.

5. The method according to claim 1, further comprising: Based on the semantic constraints of each data column in the reference editable table, a specified cell structure modification operation is performed on a specific column in the reference editable table. The specified cell structure modification operation includes: merging cells with semantic aggregation relationships, and / or splitting cells containing at least two semantic information items. In the modified specific column, the corresponding original text content is modified according to the semantic constraint relationship to generate a third type of editable table, and content validation tags are configured for the third type of editable table; The third type of editable table is rendered into a third type of table image, and table line position labels are generated for the third type of table image based on the cell position information of the third type of editable table; The third type of table image and its corresponding table line position labels are used as training samples for the table line segmentation model, and the third type of editable table and its corresponding content verification labels are used as training samples for the table content verification model.

6. The method according to claim 1, wherein obtaining the reference editable table comprises: Obtain reference table images related to the target application scenario; The reference table image is segmented using the table line segmentation model to obtain the table line position information in the reference table image; The position information of the intersection between the horizontal and vertical lines is calculated using the table line position information, and the cell position information in the reference table image is determined based on the intersection position information; Optical character recognition is performed on the reference table image to obtain the original text content and the corresponding original text content location information; Based on the matching relationship between the cell location information and the original text content location information, the original text content is associated with the corresponding cell to generate a reference editable table corresponding to the reference table image.

7. The method according to claim 1, wherein the first type of editable table is saved as an HTML format file; The step of rendering the first type of editable table into a first type of table image includes: The browser loads and renders the HTML file, and displays the rendering result in the browser window; A screenshot operation is performed on the browser window to obtain the first type of table image.

8. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-7 by executing the executable instructions.

9. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-7.

Images