Double-flow decoder table structured knowledge processing method for knowledge question-answering system

Abstract

The invention discloses a double-flow decoder table structured knowledge processing method for a knowledge question-answering system. The method comprises the following steps: constructing a background binary image of training table image data; the method comprises the following steps: extracting a background binary image through a neural network, constructing a double-flow decoder model to process extracted binary image data, and obtaining a predicted visible line graph, an invisible line graph and table cell vertexes; designing a loss function according to the visible line graph, the invisible line graph and a table cell vertex, and optimizing parameters of a double-flow decoder; and generating a final table through the optimized double-current decoder model. According to the method, optimization is carried out on the basis of a Traceable method, and various attribute graphs of table cells are directly predicted in a semantic segmentation mode. And a visible row and column line graph, an invisible row and column line graph and a vertex graph are respectively predicted through a double-flow decoder. Meanwhile, the influence of class imbalance is reduced through combination training of Focal Loss and SegLoss transformed according to table characteristics, and more attention is paid to the integrity of the regional structure.

Description

Technical Field

[0001] This invention relates to the field of table structuring technology, and more specifically to a table-structured knowledge processing method for a dual-stream decoder used in a knowledge question-answering system. Background Technology

[0002] In intelligent question-answering systems, knowledge base construction, and large-scale model enhancement applications, tabular information in document images, as an important carrier of structured knowledge, is increasingly becoming a key object of knowledge processing. Especially in professional fields such as laws and regulations, scientific research papers, financial auditing, and energy reports, the information carried by tables often possesses high structural and professional semantic value. Therefore, high-precision parsing of table structures from images has become an important preliminary step in supporting structured knowledge processing in knowledge-based question-answering systems.

[0003] Table Structure Recognition (TSR) aims to convert input table images into editable representations with logical structure (such as HTML, Excel, or JSON-based structured data) to facilitate downstream tasks such as information retrieval, knowledge fusion, and semantic reasoning. Despite continuous advancements in document image digitization technology, table images still present significant challenges in terms of layout complexity, language diversity, and image quality, resulting in considerable room for improvement in the accuracy and robustness of structured recognition.

[0004] Currently, the main forms of table structure representation include: (1) table row, column, and cell vertex coordinates; (2) table structure label description; and (3) cell boundaries and their inter-cell relationships modeling. Correspondingly, mainstream TSR methods have also evolved into three categories: Image-to-text methods model table structure recognition as an image-to-sequence conversion problem, directly generating structured tags (such as HTML). These methods often employ an encoder-decoder architecture, first generating the logical structure and then fusing it with physical boundaries to form the final structural representation. A typical example is the IEDD architecture, which uses a dual-stream decoder to generate logical tags and physical boundaries separately, combined with OCR to achieve semantic completion of cells, enhancing the completeness of the structural reconstruction. Subsequent methods such as VAST and DRCC further strengthen the visual context modeling and alignment capabilities during the decoding stage.

[0005] Cell graph-based methods: These methods focus on detecting table cells and modeling their logical relationships. They typically use graph neural networks (such as GNN and GAT) to infer spatial and semantic edges between cells, reconstructing the logical structure of the table. For example, TabStructNet and FLAG-Net establish cell graphs, improving the accuracy of structure recognition through graph reasoning.

[0006] Detection and segmentation-based methods: Based on visual segmentation and object detection, these methods detect physical features such as separator lines and cell boundaries in tables, and then combine rules or models to restore the structure. RobusTabNet introduces a spatial convolution module, which effectively improves the robustness of separator line detection. In addition, methods such as Tracetable model vertex and row / column line detection as a dense image prediction task, directly inferring structural relationships from the predicted point and line maps.

[0007] As mentioned above, deep learning has three main approaches in the field of table structure recognition. However, existing methods still have certain limitations. Detection-based methods lack robustness to changes in table layout and low-quality images, and have limited ability to model complex tables (such as nested cells or tables spanning multiple rows and columns). Image-to-text methods are limited by the length of the text output, meaning they can only recognize parts of very long tables, and their performance in complex nested tables is generally poor. Tracetable directly predicts the vertices and lines of table cells, relying heavily on visual features. It performs very well in wired tables, but its performance in wireless tables is only average. Furthermore, because its loss function is primarily L1, it is insensitive to structure and boundaries, leading to somewhat blurry predictions. Therefore, designing a novel table structure recognition method that is unrestricted in output, robust in performance, and capable of performing well in various complex long tables, wired tables, and wireless tables, and applicable to the processing of massive amounts of table data in knowledge bases, remains a pressing technical problem to be solved. Summary of the Invention

[0008] The dual-stream decoder table-structured knowledge processing method for knowledge question answering systems proposed in this invention can at least solve one of the technical problems in the background art.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for processing tabular structured knowledge using a dual-stream decoder in a knowledge question-answering system, comprising the following steps performed via a computer device: S100. Construct the background binary image of the training table image data; S200. Extract the background binary image through a neural network, construct a dual-stream decoder model to process the extracted binary image data, and obtain the predicted visible line graph, invisible line graph and table cell vertices; S300. Design a loss function based on the visible line graph, the invisible line graph, and the vertices of the table cells, and optimize the parameters of the dual-stream decoder. S400: Generate the final table using the optimized dual-stream decoder model.

[0010] Further, the method for constructing a background binary image of the training table image data in step S100 of the present invention includes: S110. Obtain the coordinates of all vertices of the table image, generate a single-channel image with the same size as the table image, and set all pixels within a radius of 5 pixels to 1, and set all other pixels in the image to zero, with the vertex coordinates as the center. S120. Obtain the coordinates of all visible horizontal lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels above and below the horizontal line to 1, and set all other pixels to zero; if there are vertices in the horizontal line, set the position of the vertex to zero as well. S130. Obtain the coordinates of all visible vertical lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels to the left and right of the vertical line position to 1, and set all other pixels to zero; if there is a vertex in the vertical line, set the position of the vertex to zero as well. S140. Obtain the coordinates of all invisible horizontal lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels above and below the horizontal line to 1, and set all other pixels to zero; if there are vertices in the horizontal line, set the position of the vertex to zero as well. S150. Obtain the coordinates of all invisible vertical lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels to the left and right of the vertical line as the center, and set all other pixels to zero; if there is a vertex in the vertical line, set the vertex position to zero as well. S160. Overlay the five images obtained in steps 1-5 and invert them to obtain a background binary image. At this time, the table lines are 0 and the background is 1.

[0011] Furthermore, step S200 of the present invention, the dual-stream decoder model construction method, includes: The model as a whole adopts an encoding and decoding structure, including an encoder, a visible line decoder, and an invisible line and vertex decoder; The encoder connects the visible line decoder and the invisible line and vertex decoder. It is responsible for inputting the visible horizontal and vertical line data into the visible line decoder, and the invisible horizontal, invisible vertical and vertex data into the invisible line and vertex decoder.

[0012] Furthermore, the loss function construction method in step S300 of the present invention includes: The overall loss function is constructed by combining Segloss and Focal loss:

[0013] in, It is the overall loss function. Segloss, This represents Focal loss.

[0014] Furthermore, the specific formula for Focal Loss in this invention is as follows: Focal Loss is an improvement on standard cross-entropy.

[0015] in, Representative category, Represents the first in the one-hot tag Is the class a positive class (1 / 0)? Representing the The model predicts the probability (after the softmax layer), with the label being 1. ); ≥0 represents a focus factor, which controls the weighting of "difficult samples"; Class weights control class imbalance; the background pixel class is set to 0.00001, and all other types are set to 1.

[0016] Furthermore, the specific formula for Segloss in this invention is as follows: Segloss only uses the structure-aware portion of the data, applying it to the four categories of visible and invisible horizontal and vertical lines.

[0017] Where i is the current pixel, Representing the neighborhood, Representing pixels Neighboring pixels in middle, Represents weight or edge weight. Represents the predicted probability. Represents the GT tag value. This represents the set of all pixels involved in the calculation.

[0018] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0019] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0020] As can be seen from the above technical solution, this invention mainly relates to the task of table image structure recognition, that is, parsing table images into an editable format with logical structure. To address the problems of poor robustness and mediocre recognition performance of current table parsing methods for complex wireless table structures, this invention optimizes the Tracetable method by directly predicting the vertex graph, visible + invisible row graph, and visible + invisible column graph of table cells through semantic segmentation. Furthermore, it uses a dual-stream decoder to predict the visible and invisible row / column graphs and vertex graph, respectively. Simultaneously, this invention reduces the impact of class imbalance through a combination of Focal Loss and SegLoss training, paying greater attention to the integrity of the region structure. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a schematic diagram of the dual-stream decoder model structure of the present invention; Figure 3 This is a table image illustration; Figure 4 This is a schematic diagram of the target image at the vertex of the table; Figure 5 This is a schematic diagram of the visible line target image in the table; Figure 6 This is a schematic diagram of a target image where the lines are not visible in the table. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0023] like Figure 1 As shown in this embodiment, the dual-stream decoder table-structured knowledge processing method for a knowledge question-answering system includes the following steps: S100. Construct the background binary image of the training table image data; S200. Extract the background binary image through a neural network, construct a dual-stream decoder to process the extracted binary image data, and obtain the predicted visible line image, invisible line image and table cell vertices; S300. Design a loss function based on the visible line graph, the invisible line graph, and the vertices of the table cells, and optimize the parameters of the dual-stream decoder. S400 generates the final table using the optimized dual-stream decoder.

[0024] The following provides a detailed explanation of the steps: S100. Construct the background binary image of the training table image data; This scheme adopts a point-to-point dense prediction task and performs prediction through semantic segmentation. The prediction target is different from existing methods, and the background binary image of the training data needs to be constructed in advance. The specific construction method is as follows: S110. Obtain the coordinates of all vertices of the table image, generate a single-channel image of the same size as the table image, and set all pixels within a radius of 5 pixels to 1, and set all other pixels in the image to zero, with the vertex coordinates as the center.

[0025] S120. Obtain the coordinates of all visible horizontal lines in the table image, generate a single-channel image of the same size as the table image, and set all pixels within 2 pixels above and below the horizontal line to 1, and set all other pixels to zero. If there are vertices in the horizontal line, set the position of the vertex to zero as well.

[0026] S130. Obtain the coordinates of all visible vertical lines in the table image, generate a single-channel image of the same size as the table image, and set all pixels within 2 pixels to the left and right of the vertical line to 1, and set all other pixels to 0. If there are vertices in the vertical line, set the position of the vertex to 0 as well.

[0027] S140. Obtain the coordinates of all invisible horizontal lines in the table image, generate a single-channel image of the same size as the table image, and set all pixels within 2 pixels above and below the horizontal line to 1, and set all other pixels to zero. If there are vertices in the horizontal line, set the position of the vertex to zero as well.

[0028] S150. Obtain the coordinates of all invisible vertical lines in the table image, generate a single-channel image of the same size as the table image, and set all pixels within 2 pixels to the left and right of the vertical line to 1, and set all other pixels to zero. If there are vertices in the vertical line, set the position of the vertex to zero as well.

[0029] S160. Overlay the five images obtained in steps 1-5 and invert them to obtain a background binary image. At this time, the table lines are 0 and the background is 1.

[0030] S200. Extract the background binary image through a neural network, construct a dual-stream decoder model to process the extracted binary image data, and obtain the predicted visible line graph, invisible line graph and table cell vertices; The dual-stream decoder processes six binary images (the target images learned by the neural network, i.e., the ideal output of the dual-stream decoder is consistent with these six images), each containing visible horizontal lines, visible vertical lines, invisible horizontal lines, invisible vertical lines, vertices, and background information.

[0031] Among them, visible lines are obvious visual features that are easier to predict, and the feature information of visible lines is located in shallower neural networks. Invisible lines and vertices, on the other hand, are semantic information that is more difficult to understand and summarize, and can only be effectively extracted through deeper neural networks.

[0032] like Figure 1 As shown, the model as a whole adopts an encoding and decoding structure, including an encoder, a visible line decoder, and an invisible line and vertex decoder; The encoder connects the visible line decoder and the invisible line and vertex decoder. It is responsible for inputting the visible horizontal and vertical line data into the visible line decoder, and the invisible horizontal, invisible vertical and vertex data into the invisible line and vertex decoder.

[0033] The encoder is responsible for extracting surface visual features and high-level semantic features of the image. It sets up a dual-stream decoder: a visible line decoder to predict visible horizontal and vertical lines, and an invisible line and vertex decoder to predict invisible horizontal, vertical, and vertices.

[0034] The output features of the encoder are fed into the visible line decoder, and the deep features are fed into the invisible line + vertex decoder. At the same time, a skip connection (dashed line in the figure) is added between the visible line decoder and the encoder to compensate for the loss of spatial information such as edges caused by encoding and decoding.

[0035] S300. Design a loss function based on the visible line graph, the invisible line graph, and the vertices of the table cells, and optimize the parameters of the dual-stream decoder. 3) Model Training While semantic segmentation predicts all rows, columns, and vertices in a table, the number of rows, columns, vertices, and background pixels differs significantly. Therefore, we employ a combination of Segloss and Focal loss to construct the overall loss function of this invention.

[0036] in, It is the overall loss function. Segloss, This represents Focal Loss. Proposed by Facebook AI, Focal Loss was initially used to address the extreme imbalance between foreground and background in object detection. It is also applicable to semantic segmentation tasks with imbalanced classes and uneven distribution of easy and difficult samples. SegLoss, through a structure-aware semantic segmentation loss function, aims to simultaneously optimize semantic consistency and structural continuity, making it particularly suitable for table-like images.

[0037] Focal Loss is an improvement on standard cross-entropy, and the specific formula is as follows:

[0038] in, Representative categories: This invention includes a total of 6 categories. Represents the first in the one-hot tag Is the class a positive class (1 / 0)? Representing the The model predicts the probability (after the softmax layer), with the label being 1. ). ≥0 represents the focus factor, which controls the weighting of "difficult samples" (set to 2.0 in this invention). Class weights control class imbalance; the background pixel class is set to 0.00001, and all other types are set to 1.

[0039] Segloss only uses the structure-aware part and is only applied to the four categories of visible and invisible horizontal and vertical lines. The specific formula is as follows:

[0040] Where i is the current pixel, Representing the neighborhood, Representing pixels Neighboring pixels in middle, Represents weight or edge weight. Represents the predicted probability. This represents the GT tag value (usually 0 or 1). This represents the set of all pixels involved in the calculation. The goal of model training is to make... Minimize as much as possible. Finally, the model updates its parameters using the Adam optimizer. It should be noted that, given the characteristics of lines, this invention has a special definition for the neighborhood of segloss (the original neighborhood includes all directions): for horizontal lines, the neighborhood represents a range of 10 pixels to the left and right in the horizontal direction, while for vertical lines, the neighborhood represents a range of 10 pixels up and down in the vertical direction.

[0041] S400 generates the final table using the optimized dual-stream decoder.

[0042] like Figure 2 As shown, the specific implementation process is as follows: 1. Training of the table structure recognition model 1) Image preprocessing to enhance data diversity: Randomly add noise to the image; randomly adjust brightness; perform random perspective transformation; adjust image size. Specific formulas are: Random noise injection: By adding random perturbations such as Gaussian noise and salt-and-pepper noise to image pixels, simulate noise pollution in real-world scenes and enhance the model's anti-interference ability. Random brightness adjustment: Adjust brightness via linear scaling or gamma correction; linear adjustment. ,in Gain coefficient , For bias , These are the original pixel values. These are the adjusted pixel values. This operation simulates table images under different lighting conditions, enhancing the model's adaptability to changes in light. Random perspective transformation: Geometrically deforms the image by randomly generating a perspective matrix.

[0043]

[0044] in These are the original pixel coordinates. The transformed coordinates, matrix parameters The matrix parameters are randomly sampled to simulate shooting angle deviations. Image size standardization: The preprocessed image is uniformly adjusted to a fixed size. A commonly used bilinear interpolation algorithm formula is:

[0045] in Distance in the original image Smooth scaling is achieved using the four nearest pixel coordinates.

[0046] (1) Obtain the longest side of the original input table image new_h = max(h,w), where h and w are the height and width of the original input table image respectively, and generate a black image img_new where all three RGB channels of new_h*new_h are 0. (2) Replace the positions of img_new from 0 to h and from 0 to w with the values ​​of the original input table image ((0,0) represents the top left corner of the image). (3) Scale the image obtained in (2) to 1280*1280. 2) Target image preprocessing: Image projection transformation (operation details remain consistent with the corresponding original table image); Image resizing (operation details remain consistent with the corresponding original table image). 3) Model training: Set hyperparameters such as learning rate, batch size, and epochs, and start training. Table structure recognition model reasoning 1) Image size adjustment: (1) Get the longest side of the original input table image new_h = max(h,w), where h and w are the height and width of the original input table image respectively, and generate a black image img_new with all three RGB channels of new_h*new_h set to 0. (2) Replace the positions of img_new from 0 to h and from 0 to w with the values ​​of the original input table image ((0,0) represents the top left corner of the image). (3) Scale the image obtained in (2) to 1280*1280. 3) Model inference to acquire five target single-channel images. Example of a target image is shown below, such as Figure 3 The table image shown, the table target vertex image is as follows Figure 4 As shown in the table; the visible line target image is as follows. Figure 5 As shown (this example image contains both visible horizontal and vertical lines, therefore the table image has no visible vertical lines, and thus no vertical lines in the image); the table's invisible lines target image is as follows. Figure 6 As shown (this example image includes both invisible horizontal and vertical lines) 4) The five output target single-channel images are thresholded to obtain binary images. The logical structure of the table is deduced based on the vertices, horizontal lines, and vertical lines contained in the images. The table structure is then reconstructed using a merging strategy, and finally, the HTML of the table structure is generated. (This step is not the focus of this invention; the post-processing method of Tracetable is directly applied, and will not be described in detail here.) Figure 3 Here is an HTML example of a table structure for a table image: Sample T c (K) d (nm) d ac (nm) z0(nm) l ab (10 K) (nm) l ab (0 K) (nm) Δ0(meV) YBCO film 90.2(2) 150(8) 119(1) 0(2) 161(1) 158(1) 22(2) YBCO x film 90.8(1) 152(8) 134(1) 1(2) 154(1) 150(1) 24(2) YBCO bulk

[23] 94.1(1) 152(8) 134(1) 10.3(5) 115(4) 110(1) 20(4) In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0047] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the dual-stream decoder table-structured knowledge processing methods for knowledge question-answering systems described above.

[0048] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0049] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0051] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0052] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing tabular structured knowledge using a dual-stream decoder in a knowledge question-answering system, characterized in that, Perform the following steps using a computer device: S100. Construct the background binary image of the training table image data; S200. Extract the background binary image through a neural network, construct a dual-stream decoder model to process the extracted binary image data, and obtain the predicted visible line graph, invisible line graph and table cell vertices; S300. Design a loss function based on the visible line graph, the invisible line graph, and the vertices of the table cells, and optimize the parameters of the dual-stream decoder. S400: Generate the final table using the optimized dual-stream decoder model.

2. The dual-stream decoder table-structured knowledge processing method for a knowledge question-answering system according to claim 1, characterized in that, Step S100, the method for constructing the background binary image of the training table image data, includes: S110. Obtain the coordinates of all vertices of the table image, generate a single-channel image with the same size as the table image, and set all pixels within a radius of 5 pixels to 1, and set all other pixels in the image to zero, with the vertex coordinates as the center. S120. Obtain the coordinates of all visible horizontal lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels above and below the horizontal line to 1, and set all other pixels to zero; if there are vertices in the horizontal line, set the position of the vertex to zero as well. S130. Obtain the coordinates of all visible vertical lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels to the left and right of the vertical line position to 1, and set all other pixels to zero; if there is a vertex in the vertical line, set the position of the vertex to zero as well. S140. Obtain the coordinates of all invisible horizontal lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels above and below the horizontal line to 1, and set all other pixels to zero; if there are vertices in the horizontal line, set the position of the vertex to zero as well. S150. Obtain the coordinates of all invisible vertical lines in the table image, generate a single-channel image of the same size as the table image, and set all outer pixels within 2 pixels to the left and right of the vertical line as the center, and set all other pixels to zero; if there is a vertex in the vertical line, set the vertex position to zero as well. S160. Overlay the five images obtained in steps 1-5 and invert them to obtain a background binary image. At this time, the table lines are 0 and the background is 1.

3. The dual-stream decoder table-structured knowledge processing method for a knowledge question-answering system according to claim 1, characterized in that, The S200 dual-stream decoder model construction method includes: The model as a whole adopts an encoding and decoding structure, including an encoder, a visible line decoder, and an invisible line and vertex decoder; The encoder connects the visible line decoder and the invisible line and vertex decoder. It is responsible for inputting the visible horizontal and vertical line data into the visible line decoder, and the invisible horizontal, invisible vertical and vertex data into the invisible line and vertex decoder.

4. The dual-stream decoder table-structured knowledge processing method for a knowledge question-answering system according to claim 1, characterized in that, The loss function construction method in step S300 includes: The overall loss function is constructed by combining Segloss and Focal loss: in, It is the overall loss function. Segloss, This represents Focal loss.

5. The dual-stream decoder table-structured knowledge processing method for a knowledge question-answering system according to claim 4, characterized in that, The specific formula for Focal Loss is as follows: Focal Loss is an improvement on standard cross-entropy. in, Representative category, Represents the first in the one-hot tag Is the class a positive class (1 / 0)? Representing the The model predicts the probability (after the softmax layer), with the label being 1. ); ≥0 represents a focus factor, which controls the weighting of "difficult samples"; Class weights control class imbalance; the background pixel class is set to 0.00001, and all other types are set to 1.

6. The dual-stream decoder table-structured knowledge processing method for a knowledge question-answering system according to claim 4, characterized in that, The specific formula for Segloss is as follows: Segloss only uses the structure-aware portion of the data, applying it to the four categories of visible and invisible horizontal and vertical lines. Where i is the current pixel, Representing the neighborhood, Representing pixels Neighboring pixels in middle, Represents weight or edge weight. Represents the predicted probability. Represents the GT tag value. This represents the set of all pixels involved in the calculation.

Images