Relational Table Data Learning System Based on Intra-Table and Inter-Table Interactions

Abstract

A relational table data learning system based on intra-table and inter-table interactions includes: a column-aware table encoder module, an intra-table interaction learning module, an inter-table interaction learning module, an intra-table and inter-table fusion and output module, and a training optimization module. This invention significantly improves the representation ability of different types of related information and reduces the attention computation overhead in the relational table data learning process during intra-table interactions by coordinating intra-table and inter-table interactions under a unified learning framework, thereby improving the processing efficiency in large-scale data scenarios.

Description

Technical Field

[0001] This invention relates to a technology in the field of neural network applications, specifically a relational table data learning system based on intra-table and inter-table interactions. Background Technology

[0002] Relational Database Management Systems (RDBMS) are widely used in scenarios such as financial risk control, recommendation systems, enterprise business analysis, healthcare, and government affairs. This type of data is typically distributed across multiple tables, forming a stable semantic association structure through primary key-foreign key (PK-FK) relationships. When performing machine learning modeling on multi-table relational data, the model needs to handle both intra-table interactions and inter-table interactions based on PK-FK. Existing solutions generally suffer from problems such as complex feature engineering, susceptibility to damage to the original relational structure, high computational overhead, susceptibility to noise interference, and limited processing efficiency in large-scale data scenarios. Summary of the Invention

[0003] This invention addresses the shortcomings of existing multi-table learning techniques, which rely on denormalization or general message passing, easily leading to loss of relational structure information, cross-table noise propagation, and excessive computational complexity. It proposes a relational table data learning system based on intra-table and inter-table interactions. By coordinating intra-table and inter-table interactions within a unified learning framework, it can significantly improve the representation capability of different types of related information and reduce the attention computation overhead in the relational table data learning process during intra-table interactions, thereby improving processing efficiency in large-scale data scenarios.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a relational table data learning system based on intra-table and inter-table interactions, comprising: a column-aware table encoder module, an intra-table interaction learning module, an inter-table interaction learning module, an intra-table and inter-table fusion and output module, and a training optimization module. Specifically: the column-aware table encoder module independently learns row-level dense representations for each table, obtaining initial dense representations for each row in each table; the intra-table interaction learning module performs intra-table interaction learning processing on the initial dense representations based on the association information between rows within the same table, obtaining enhanced intra-table representations; the inter-table interaction learning module performs inter-table interaction learning processing on related rows between different tables based on primary key-foreign key relationships, obtaining enhanced inter-table representations; the intra-table and inter-table fusion and output module fuses the enhanced intra-table and inter-table representations to obtain the final representation of the target row, and outputs the corresponding prediction result based on the final representation; the training optimization module employs supervised end-to-end training to obtain a trained relational table data learning model.

[0006] This invention relates to an application based on the above system, which represents relational table data as a set of tables and a set of PK-FK relations between tables as tuples. The dependencies in the relation table are clearly divided into intra-table interactions and inter-table interactions. The two types of interaction results are obtained by using intra-table context modeling with an scalable linear attention mechanism and cross-table information supplementation with a message passing mechanism based on PK-FK graphs. Finally, the results are fused and then classified or regressed downstream to improve the representation ability of different types of related information.

[0007] Technical effect

[0008] This invention improves the utilization of key information columns by assigning importance weights to different columns through a column-aware table encoder, thereby reducing the interference of irrelevant column information on model learning. It reduces the computational complexity and storage overhead of attention through a linear attention mechanism for interactive learning, thereby improving the processing efficiency and scalability in large-scale relational table data scenarios. During inter-table interactions, it introduces structural constraints based on primary key-foreign key relationships to limit the propagation of cross-table information to related records, thereby reducing the noise interference introduced by irrelevant rows. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the system of the present invention;

[0010] Figure 2 This is a flowchart of an example implementation. Detailed Implementation

[0011] like Figure 1 As shown in the figure, this embodiment relates to a relational table data learning system based on intra-table and inter-table interactions, including: a column-aware table encoder module, an intra-table interaction learning module, an inter-table interaction learning module, an intra-table and inter-table fusion and output module, and a training optimization module.

[0012] The column-aware table encoder module includes: a row-level representation generation unit, a column summary generation unit, and a column weight calculation unit. Specifically: the row-level representation generation unit encodes each row of data in the preprocessed tables to obtain an initial dense representation for each row; the column summary generation unit aggregates each column based on its representation in each row to obtain a column summary vector for each column; the column weight calculation unit calculates the importance score for each column based on its column summary vector and normalizes the importance score to obtain the column weight for each column; and the column weights weight the initial dense representations of each row output by the row-level representation generation unit to obtain a column-aware row-level representation.

[0013] The in-table interaction learning module includes an in-table linear attention interaction unit and an in-table representation update unit. The in-table linear attention interaction unit learns the information between rows in the same table based on the in-table interaction relationship and uses a linear attention mechanism to learn the interaction features. The in-table representation update unit is connected to the in-table linear attention interaction unit and fuses the in-table interaction features with the input column-aware row-level representation to obtain the in-table enhanced representation.

[0014] The aforementioned linear attention mechanism refers to: ,in: This represents the in-table interaction output obtained from the linear attention computation of the l-th layer. Q, K, and V represent the query matrix, key matrix, and value matrix after Frobenius normalization, respectively. It is an n×n matrix of all 1s. It is a diagonal normalized matrix. It is a vector of all 1s. This is a diagonalization operation.

[0015] The inter-table interaction learning module includes: a relationship graph construction unit, a neighbor sampling unit, a cross-table aggregation unit, and an inter-table representation update unit. Specifically: the relationship graph construction unit constructs an inter-table relationship graph based on the primary key-foreign key correspondences between different relationship tables; the neighbor sampling unit samples neighbor row records with primary key-foreign key associations from the inter-table relationship graph for each target row record; the cross-table aggregation unit aggregates the representations of the sampled neighbor row records and combines them with the target row record's own representation to generate inter-table interaction features; and the inter-table representation update unit updates the representation of the target row record based on the inter-table interaction features to obtain an enhanced inter-table representation.

[0016] In the table relationship graph, nodes represent row records in the relationship table, and edges represent the connection relationships between row records with primary key-foreign key associations.

[0017] The cross-table aggregation unit employs, but is not limited to, the techniques described by Scarselli et al. in “The Graph Neural Network Model” (IEEE Transactions on Neural Networks, 2009) and / or Hamilton et al. in “Inductive Representation Learning on Large Graphs” (NeurIPS 2017, GraphSAGE).

[0018] The training optimization module includes a label acquisition unit, a loss construction unit, and a parameter update unit. The label acquisition unit collects the real labels corresponding to the prediction results from the task output unit. The loss construction unit constructs a supervised learning loss function based on the prediction results and the real labels. The parameter update unit performs end-to-end joint optimization of the parameters in the column-aware encoder module, the intra-table interaction learning module, the inter-table interaction learning module, and the intra-table-inter-table fusion and output module through backpropagation, thereby achieving relational table data learning.

[0019] like Figure 2 The image shows an application of the system described above in this embodiment, including:

[0020] Step 1: Independently learn row-level dense representations for each table, specifically including:

[0021] 1.1 Setting up a table Column, number Listed as , No. Line number The column cell value is Pre-encoding mapping is performed on columns of different types separately. Map the cells to the same d-dimensional vector space: Embed vectors in cells, pre-encode mapping The implementation method can be selected according to the column type.

[0022] For example, learnable embedding tables are used for categorical features, linear transformations and normalization are used for numerical features, and lightweight encoders (such as average pooling or multilayer perceptrons) are used for text / sequence features to obtain fixed-length vectors. For missing values, missing value embeddings or missing value indicator features can be introduced to improve robustness. This yields the column matrix of the j-th column. .

[0023] 1.2 Considering the varying importance of different tasks to different columns, importance weights are assigned to each column, enabling the relational table data learning system to automatically focus on columns with higher information content. This is achieved by performing average pooling on each column to obtain a column summary vector. ,in: Indicates the first The number of samples corresponding to the column Indicates the sample index. Indicates the first The sample at the th The feature representation corresponding to the column, Indicates the first The column summary vector obtained by average pooling.

[0024] 1.3 Calculate column scores based on summary vectors And normalize to obtain column weights ,in: This represents the column score of column j. This represents averaging the elements of each dimension of the vector. Indicates the first Normalized weights of columns, This represents an exponential function.

[0025] Step 2: In the fusion phase, each column is recalibrated according to its weight, and after column concatenation, a Residual Network (ResNet) is used to capture high-order interactions across columns to obtain the row-level representation of table T. , For splicing operations, It is a deep residual network.

[0026] Step 3: Target table , and output its pre-encoded mapping As the initial row representation for in-table interactions: To clarify The iteration method, in the first... In the layer, firstly, for Perform linear projection to construct the query, key, and value matrix: , , For the first Layer learnable matrix, For layer normalization, the subscript intra indicates intra-table interaction, and (0) represents layer 0.

[0027] To control the numerical range of attention scores, Frobenius normalization is performed on the query matrix Q, key matrix K, and value matrix V, specifically as follows: ,in: This represents the Frobenius norm.

[0028] Step 4: Employ an attention mechanism to characterize the relationships between different rows within the same table, enabling each row to aggregate contextual information from other rows in the same table to achieve intra-table interaction. Specifically: Let the first row be... Layer input is We construct Q, K, and V and obtain them through linearization calculations: , It is a matrix of all ones. It is a vector of all 1s. This is a diagonalization operation.

[0029] The linearization computation refers to the computable expression obtained by approximating the Softmax attention using a first-order Taylor expansion, thus avoiding the explicit construction of an n×n matrix. Intra-table interactions can be stacked in multiple layers, with layer normalization, residual connections, and feedforward networks introduced after each layer to enhance nonlinear representation; simultaneously, a multi-head mechanism can be used to model different semantic subspaces.

[0030] Step 5: Modeling inter-table interactions based on message passing in a PK-FK graph: After treating the target table row and the auxiliary table row as the node sets V and U on both sides of a bipartite graph, respectively, the PK-FK connections are induced into an edge set E to obtain the bipartite graph. In multi-table scenarios, each table row can be considered a heterogeneous node, and different PK-FK relationship types can be considered different edge types, thus forming a heterogeneous graph, which specifically includes:

[0031] 5.1 Target Table Each row is mapped to a target node in the graph. ,Will Indicates the first The target node v at the layer level, i.e., the d-dimensional row representation of a row in the target table, is to represent all... Stacking rows together yields a matrix-style representation of the tables. .

[0032] 5.2 Map each row in the auxiliary table to a node u in the auxiliary node set U. For any target node... In its first Layered table representation The row and column representation is updated as follows: , Let V be the set of neighbors of the target node v. Let be the row representation of node u; AGG is the weighted summation or attention-weighted summation aggregation function; UPD is the linear layer, nonlinear, or gated structure update function.

[0033] 5.3 Stacking the representations of all target nodes row by row yields the first... Layered matrix representation of table When multiple PK-FK relationship types exist, different parameters are configured for different relationship types, and messages from different relationship sources are distinguished during aggregation, thereby utilizing relationship structure information in a more granular manner.

[0034] Step 6: Learn the in-table interactions Inter-layer and inter-table interactive learning The results from each layer are fused, and the fused row representation is input into a multilayer perceptron to obtain the final prediction result. Supervised learning is used to calculate the loss, and backpropagation is used to update the parameters. Specifically, the loss function is as follows: Where: Y is the true label, and the final prediction result is... , , The learnable coefficient.

[0035] In multi-task learning scenarios, different output headers can be configured for different tasks to share encoding and interaction module parameters and improve overall generalization ability.

[0036] The supervised learning mentioned refers to the following: for classification tasks, cross-entropy loss (CE) is used; for regression tasks, mean squared error (MSE) loss is used. During training, the Adam optimizer can be used, and strategies such as learning rate scheduling and early stopping can be introduced. If necessary, L2 regularization and dropout can be added to improve generalization ability, and sampling strategies can be set for different tables and different relation types to balance the distribution of training data.

[0037] To adapt to extremely large relational table data, mini-batch training and sampling can be used for acceleration. For example, a subset of rows from the target table can be sampled to form a sub-table, and a neighbor subgraph related to this sub-table can be sampled from the PK-FK graph. Linear attention and message passing are then computed on the sampled substructure, thereby further reducing training overhead without significantly degrading performance. From a time complexity perspective, the main overhead of the column-aware table encoder comes from the column fusion network, approximately... The main overhead of the in-table linear attention approximation is: The main overhead of graph-based message passing between tables is... , where e is the number of edges induced by PK-FK; therefore, the whole can achieve near-linear expansion with the number of rows n and the number of edges e.

[0038] Through practical application experiments, the relational table data learning system based on intra-table and inter-table interactions of this invention was run in a hardware environment configured with a single Nvidia RTX A6000 graphics card and 48GB of video memory. The system was trained using a training set, validation set, and test set partitioning method, with end-to-end training conducted through supervised learning. Cross-entropy loss was used for classification tasks, and L1 loss was used for regression tasks. Extensive experiments were conducted on multiple publicly available relational table datasets, including the TML1M, TLF2K, and TACM12K datasets from the SJTUTables benchmark, and the rel-amazon, rel-avito, rel-event, rel-f1, rel-hm, rel-stack, and rel-trial datasets from the RelBench benchmark, covering small, medium, and large-scale relational table data and various task scenarios such as classification and regression. Experimental results show that this invention can effectively complete relational table data learning tasks and achieve good predictive performance. The performance and evaluation results of various models in the binary classification prediction task are shown in Table 1, and the quantitative evaluation results in the regression prediction task are shown in Table 2.

[0039] Table 1

[0040] Table 2

[0041] Compared to existing technologies, this invention explicitly divides dependencies in relational table learning into intra-table interactions and inter-table interactions, and designs corresponding learning modules and representation fusion mechanisms for each. This allows the model to utilize both intra-table context and cross-table supplementary information, thus better suiting the structural characteristics of PK-FK multi-table data. Furthermore, this invention employs linear attention approximation in intra-table interactions, avoiding the construction of an n×n attention matrix; and uses message passing based on the PK-FK graph in inter-table interactions, making the computational complexity nearly linearly related to the number of PK-FK induced edges e. Therefore, it remains trainable and deployable even with a large number of rows and relational scales, and can further improve efficiency by combining mini-batch and neighbor sampling. Simultaneously, this invention introduces PK-FK-connection-based mask / adjacency constraints in inter-table interactions, aggregating only connected row pairs to reduce noise propagation from irrelevant row pairs; the column-aware table encoder automatically focuses on key columns by learning column weights and missing value handling strategies, improving robustness to missing and heterogeneous features. This method features end-to-end training and is engineering-friendly, making it suitable for typical relational table application scenarios such as financial risk control, recommendation, marketing, operations, healthcare, and government affairs. It can also be flexibly configured for classification, regression, or multi-task learning according to task requirements.

[0042] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A relational table data learning system based on intra-table and inter-table interactions, characterized in that, include: The system consists of a column-aware table encoder module, an intra-table interaction learning module, an inter-table interaction learning module, an intra-table-inter-table fusion and output module, and a training optimization module. Among them, the column-aware table encoder module independently learns row-level dense representations for each table to obtain the initial dense representations of each row in each table. The intra-table interaction learning module performs intra-table interaction learning on the initial dense representation based on the relationship information between rows within the same table, resulting in an enhanced intra-table representation. The inter-table interaction learning module performs inter-table interaction learning on related rows between different tables based on the primary key-foreign key relationship, resulting in an enhanced inter-table representation. The intra-table-inter-table fusion and output module fuses the intra-table and inter-table enhanced representations to obtain the final representation of the target row and outputs the corresponding prediction result based on the final representation. The training and optimization module uses supervised learning for end-to-end training to obtain the trained relational table data learning model.

2. The relational table data learning system based on intra-table and inter-table interactions according to claim 1, characterized in that, The column-aware table encoder module includes: a row-level representation generation unit, a column summary generation unit, and a column weight calculation unit. Specifically: the row-level representation generation unit encodes each row of data in the preprocessed tables to obtain an initial dense representation for each row; the column summary generation unit aggregates each column based on its representation in each row to obtain a column summary vector for each column; the column weight calculation unit calculates the importance score for each column based on its column summary vector and normalizes the importance score to obtain the column weight for each column; and the column weights weight the initial dense representations of each row output by the row-level representation generation unit to obtain a column-aware row-level representation.

3. The relational table data learning system based on intra-table and inter-table interactions according to claim 1, characterized in that, The in-table interaction learning module includes an in-table linear attention interaction unit and an in-table representation update unit. The in-table linear attention interaction unit learns the information between rows in the same table based on the in-table interaction relationship and uses a linear attention mechanism to learn the interaction features. The in-table representation update unit is connected to the in-table linear attention interaction unit and fuses the in-table interaction features with the input column-aware row-level representation to obtain the in-table enhanced representation.

4. The relational table data learning system based on intra-table and inter-table interactions according to claim 1, characterized in that, The inter-table interaction learning module includes: a relationship graph construction unit, a neighbor sampling unit, a cross-table aggregation unit, and an inter-table representation update unit. Specifically: the relationship graph construction unit constructs an inter-table relationship graph based on the primary key-foreign key correspondences between different relationship tables; the neighbor sampling unit samples neighbor row records with primary key-foreign key associations from the inter-table relationship graph for each target row record; the cross-table aggregation unit aggregates the representations of the sampled neighbor row records and combines them with the target row record's own representation to generate inter-table interaction features; and the inter-table representation update unit updates the representation of the target row record based on the inter-table interaction features to obtain an enhanced inter-table representation. In the table relationship graph, nodes represent row records in the relationship table, and edges represent the connection relationships between row records with primary key-foreign key associations.

5. The relational table data learning system based on intra-table and inter-table interactions according to claim 1, characterized in that, The training optimization module includes a label acquisition unit, a loss construction unit, and a parameter update unit. The label acquisition unit collects the real labels corresponding to the prediction results from the task output unit. The loss construction unit constructs a supervised learning loss function based on the prediction results and the real labels. The parameter update unit performs end-to-end joint optimization of the parameters in the column-aware encoder module, the intra-table interaction learning module, the inter-table interaction learning module, and the intra-table-inter-table fusion and output module via backpropagation, thereby achieving relational table data learning.

6. An application based on the system described in any one of claims 1-5, characterized in that, Represent relational table data as a set of tables and a set of PK-FK relations between the tables as tuples. The dependencies in the relation table are clearly divided into intra-table interactions and inter-table interactions. The two types of interaction results are obtained by using intra-table context modeling with an scalable linear attention mechanism and cross-table information supplementation with a message passing mechanism based on PK-FK graphs. Finally, the results are fused and then classified or regressed downstream to improve the representation ability of different types of related information.

7. The application according to claim 6, characterized in that, specifically include: Step 1: Independently learn row-level dense representations for each table, specifically including: Step 2: In the fusion phase, each column is recalibrated according to its weight, and after column concatenation, a Residual Network (ResNet) is used to capture high-order interactions across columns to obtain the row-level representation of table T. , For splicing operations, It is a deep residual network; Step 3: Target table , and output its pre-encoded mapping As the initial row representation for in-table interactions: To clarify The iteration method, in the first... In the layer, firstly, for Perform linear projection to construct the query, key, and value matrix: , , For the first Layer learnable matrix, For layer normalization, the subscript intra indicates intra-table interaction, and (0) represents the 0th layer; Step 4: Employ an attention mechanism to characterize the relationships between different rows within the same table, enabling each row to aggregate contextual information from other rows in the same table to achieve intra-table interaction. Specifically: Let the first row be... Layer input is We construct Q, K, and V and obtain them through linearization calculations: , It is a matrix of all ones. It is a vector of all 1s. This is a diagonalization operation; Step 5: Modeling inter-table interactions based on message passing in a PK-FK graph: After treating the target table row and the auxiliary table row as the node sets V and U on both sides of a bipartite graph, respectively, the PK-FK connections are induced into an edge set E to obtain the bipartite graph. For multi-table scenarios, each table row can be considered as a heterogeneous node, and different PK-FK relationship types can be considered as different edge types, thus forming a heterogeneous graph, specifically including: Step 6: Learn the in-table interactions Inter-layer and inter-table interactive learning The results from each layer are fused, and the fused row representation is input into a multilayer perceptron to obtain the final prediction result. Supervised learning is used to calculate the loss, and backpropagation is used to update the parameters. Specifically, the loss function is as follows: Where: Y is the true label, and the final prediction result is... , , The learnable coefficient.

8. The application according to claim 6, characterized in that, Step 1 specifically includes: 1.1 Setting up a table Column, number Listed as , No. Line number The column cell value is Pre-encoding mapping is performed on columns of different types. Map the cells to the same d-dimensional vector space: Embed vectors in cells, pre-encode mapping The implementation method can be selected according to the column type; 1.2 Considering the varying importance of different tasks to different columns, importance weights are assigned to each column, enabling the relational table data learning system to automatically focus on columns with higher information content. This is achieved by performing average pooling on each column to obtain a column summary vector. ,in: Indicates the first The number of samples corresponding to the column Indicates the sample index. Indicates the first The sample at the th The feature representation corresponding to the column, Indicates the first The column summary vector obtained by average pooling; 1.3 Calculate column scores based on summary vectors And normalize to obtain column weights ,in: This represents the column score of column j. This represents averaging the elements of each dimension of the vector. Indicates the first Normalized weights of columns, This represents an exponential function.

9. The application according to claim 6, characterized in that, Step 5 specifically includes: 5.1 Target Table Each row is mapped to a target node in the graph. ,Will Indicates the first The target node v at the layer level, i.e., the d-dimensional row representation of a row in the target table, is to represent all... Stacking rows together yields a matrix-style representation of the tables. ; 5.2 Map each row in the auxiliary table to a node u in the auxiliary node set U. For any target node... In its first Layered table representation The row and column representation is updated as follows: , Let V be the set of neighbors of the target node v. Let be the row representation of node u; AGG is the weighted summation or attention-weighted summation aggregation function; UPD is the linear layer, nonlinear layer, or gated structure update function. 5.3 Stacking the representations of all target nodes row by row yields the first... Layered matrix representation of table When multiple PK-FK relationship types exist, different parameters are configured for different relationship types, and messages from different relationship sources are distinguished during aggregation, thereby making more granular use of relationship structure information.

Images